DEV Community: Aditya

Laravel API - Import 1 million records with validation in few seconds.

Aditya — Mon, 19 Feb 2024 17:05:24 +0000

Database import is a crucial element and needs to be done right. Whether it's asking a user to fill out a form or uploading a CSV file through a UI, all data that enters the database must go through proper validation. For large CSV imports, balancing security and performance can be challenging. In this article, I'll demonstrate how to import 1 million records into a MySQL database with validation using Laravel APIs.

Here is the YouTube video from my channel WebDevWithArtisan

The key is to optimize for speed while not compromising data integrity. I'll use a REST API approach so any front-end can upload files. To handle the load, we'll employ a divide-and-conquer tactic by splitting the large CSV into chunks and harnessing Laravel's powerful job batching for queues.

Validation Strategy

We need to validate the structure and data types of the CSV before insertion. I'll create validator classes for each model that will be populated. These will check for required fields, data formats, relationships, etc.

Import Workflow

The API endpoint accepts a CSV file upload
The file gets divided into smaller chunks
Each chunk is pushed into a job queue
Jobs will validate rows and batch insert into the database

Optimization Considerations

Use queue workers and supervised jobs for failure resiliency
Tune batch sizes to balance speed and memory usage
Add rate limiting to manage concurrent API requests
Transact inserts in a single database transaction for data integrity

By following these patterns, we can reliably ingest datasets of any size while protecting data quality. Now let's examine the code to implement our CSV import solution using Laravel and MySQL.

Prerequisite: This article assumes the reader has a basic understanding of Laravel and its core concepts.

Create a Laravel project with a Breeze starter kit.

As I am creating a project from scratch, I will use the Laravel Breeze API starter kit as it is minimal and comes with all the features (Laravel Sanctum configs) that we need to create a Laravel API.

If you are integrating this in your own project then you might want to skip this step as starter kits will override your current code.

Run the following command to create a Laravel project



composer create-project laravel/laravel laravelapi

Then run the following command to create Breeze API scaffolding.



composer require laravel/breeze --dev



php artisan breeze:install

The terminal output should be something similar to this

Creating model, controller and migration

I have added the Database credentials in my .env file. Let's create migrations. The fake dataset I have is page_analytics. Let's create a model, migration and controller.



php artisan make:model PageAnalytic -mc

This is the schema of my page_analytics table



<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    /**
     * Run the migrations.
     */
    public function up(): void
    {
        Schema::create('page_analytics', function (Blueprint $table) {
            $table->id();
            $table->string('url');
            $table->string('activity');
            $table->foreignId('user_id')->nullable()->constrained('users')->nullOnDelete();
            $table->timestamps();
        });
    }

    /**
     * Reverse the migrations.
     */
    public function down(): void
    {
        Schema::dropIfExists('page_analytics');
    }
};

Let's push this table to the database.



php artisan migrate

I have also seeded fake 10k users in the table using the default UserFactory that comes with the Laravel project.

Laravel Queue and Batch Setup

I will be using a queue connection of the database, you can use anything (SQS, Redis). So we will change the environment variable of QUEUE_CONNECTION in the .env file



QUEUE_CONNECTION=database

Then we will run these two commands to create the migrations file



php artisan queue:table



php artisan queue:batches-table

Then migrate these two tables to our database.



php artisan migrate

Possible PHP errors

When uploading large files in PHP you might get errors related to

Memory limit
Request Timeout
Post Content limit

To solve the file upload limit error you will need to change the configurations in php.ini file. You need to change the configurations for upload_max_filesize and post_upload_size. Make sure to have post_upload_size > upload_max_filesize.

To handle request timeout errors we will use PHP Generator and Laravel Job Batches.

Working on API

With Laravel Breeze API setup we get great scaffolding for auth. I will modify the login request function in App\Http\Controllers\Auth\AuthenticatedSessionController to have it send an API token.



<?php

namespace App\Http\Controllers\Auth;

// use statements

class AuthenticatedSessionController extends Controller
{
    /**
     * Handle an incoming authentication request.
     */
    public function store(LoginRequest $request): JsonResponse
    {
        $request->authenticate();

        $token = $request->user()->createToken('authToken')->plainTextToken;

        return response()->json([
            'token' => $token,
            'user' => $request->user(),
        ]);
    }
}

I will be using Postman to test our API endpoints. The problem is our API might throw a CSRF Cookie error for the Register and Login route. For that reason purpose we will set the csrf cookie by adding this pre-request script in Postman



pm.sendRequest({
    url: 'http://localhost:8000/sanctum/csrf-cookie',
    method: 'GET'
}, function (error, response, {cookies}) {
    if (!error){
        pm.collectionVariables.set('xsrf-cookie', cookies.get('XSRF-TOKEN'))
    }
})

After this setup, if we send a post request to the login route we should get user details and an API token. We will use this token to make further requests.

API routes

Let's add two API routes, one for checking the batch status and one for importing the CSV file into database.



<?php

use App\Http\Controllers\BatchController;
use App\Http\Controllers\PageAnalyticController;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Route;

Route::middleware(['auth:sanctum'])->group(function(){
    Route::get('/user', function (Request $request) {
        return $request->user();
    });

    Route::get("/import-status/{batch_id}", [BatchController::class, 'status']);

    Route::post("/import-analytics", [PageAnalyticController::class, 'store']);
});

API for batch status

As we are importing record using jobs which will be done in batch, we can use the batch id to get the progress of the entire import process.



//BatchController.php
<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use Illuminate\Support\Facades\Bus;

class BatchController extends Controller
{
    public function status(string $batch_id)
    {
        return response()->json([
            'details' =>  Bus::findBatch($batch_id)
        ]);
    }
}

API for file import

Now this is the real game, let's add the code below in PageAnalyticController.php



<?php

namespace App\Http\Controllers;

use App\Services\PageAnalyticsImportService;
use Illuminate\Http\Request;

class PageAnalyticController extends Controller
{
    public function store(Request $request)
    {
        $data = $request->validate([
            'file' => ['required', 'file', 'mimes:csv', 'max:61440']
        ]);

        $batchID = (new PageAnalyticsImportService())->import($data);

        return response()->json(['batch_id' => $batchID], 201);
    }
}

I am validating the file for CSV mime and the max upload size of 60 MB.

I have also created a PageAnalyticsImportService service in app/Services to keep the controller code cleaner. It's a PHP class which will handle the import process.



<?php

namespace App\Services;

use App\Jobs\AnalyticsImportJob;
use Generator;
use Illuminate\Support\Facades\Bus;

class PageAnalyticsImportService
{
    public function import(array $data): string
    {
        $batch = Bus::batch([])->dispatch();

        $filePath = $data['file']->getRealPath();

        foreach($this->chunkAsGenerator($filePath) as $chunk)
        {
            $batch->add(new AnalyticsImportJob($chunk));
        }

        return $batch->id;
    }

    public function chunkAsGenerator(string $filePath) : Generator
    {
        $handle = fopen($filePath, 'r');

        if($handle !== false)
        {
            fgetcsv($handle, 0, ',');

            $chunkData = [];
            $chunkSize = 0;

            while(($row = fgetcsv($handle, 0, ',')) !== false)
            {
                $chunkData[] = $row;
                $chunkSize++;
                if($chunkSize >= 500)
                {
                    yield $chunkData;
                    $chunkData = [];
                    $chunkSize = 0;
                }
            }

            if(!empty($chunkData))
            {
                yield $chunkData;
            }

            fclose($handle);
        }

    }
}

Let's break down this code.

import(): This method orchestrates the import process and is designed to handle large CSV files efficiently without overwhelming the server's resources. The method starts by creating an empty batch and dispatching it. It accepts the validated request input and extracts the uploaded file's temporary path. The chunkAsGenerator function is called with the file path as its argument. This generator function reads the CSV file in chunks, yielding each chunk as it goes. This approach is key to managing memory usage effectively, allowing the server to handle large files without running into memory limit issues. For each chunk yielded by the generator, a new AnalyticsImportJob is instantiated with the chunk data and added to the previously created batch. This design ensures that the heavy lifting of processing CSV data is offloaded to Laravel's queue system, allowing for asynchronous processing and better resource management. Finally, the import method returns the batch's id. This ID can be used to monitor the batch's progress, check for completion, or handle failures, providing flexibility in managing the import process.
chunkAsGenerator(): At the heart of the import process is the chunkAsGenerator method. This method cleverly breaks down the CSV file into manageable chunks, reducing the risk of exhausting the server's memory. The method attempts to open the provided CSV file path for reading. If successful, it proceeds; otherwise, it exits. The first call to fgetcsv reads and ignores the CSV header row. The method then enters a loop, reading each row of the CSV file. Rows are accumulated into an array ($chunkData) until the specified chunk size is reached (in this case, 500 rows which can be changed as needed). Once the limit is hit, the current chunk is yielded, and the accumulation starts anew for the next chunk. After the loop, if there’s any remaining data that didn't meet the chunk size threshold, it's yielded as the final chunk. This ensures no data is left behind. Finally, the file handle is closed, ensuring no resources are left hanging.

Creating the job

AnalyticsImportJob is a Laravel job. It is designed to process chunks of CSV data related to page analytics asynchronously. It's another crucial component of the system that handles the efficient insertion of large volumes of data into the database, ensuring data integrity and validation.

Run the command below to create the job for import



php artisan make:job AnalyticsImportJob

Job code



<?php

namespace App\Jobs;

use App\Models\PageAnalytic;
use App\Models\User;
use Illuminate\Bus\Batchable;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Validator;

class AnalyticsImportJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Batchable;

    public array $chunkData;
    /**
     * Create a new job instance.
     */
    public function __construct(array $chunk)
    {
        $this->chunkData = $chunk;
    }

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        $validatedInputs = $this->validateEachRowAndDiscard();
        // Store the validated inputs in the database
        PageAnalytic::insert($validatedInputs);
    }

    public function validateEachRowAndDiscard()
    {
        $validatedInputs = [];
        $size = count($this->chunkData);

        for($i=0; $i<$size; $i++)
        {
           $inputArray = [
            'user_id' => $this->chunkData[$i][0],
            'created_at' => $this->chunkData[$i][1],
            'activity' => $this->chunkData[$i][2],
            'url' => $this->chunkData[$i][3],
           ];

           $validator = Validator::make($inputArray, [
                'user_id' => ['required', 'integer'],
                'activity' => ['required', 'string'],
                'url' => ['required', 'string'],
           ]);

            if($validator->fails())
            {
                continue;
            }  

            $validatedInputs[] = $inputArray;
        }

        $userIDsArray = array_column($validatedInputs, 'user_id');
        $uniqueUserIDs = array_unique($userIDsArray);

        $userIDsFromDB = User::whereIn('id', $uniqueUserIDs)->pluck('id')->toArray();

        $validatedInputs = array_filter($validatedInputs, fn($input) => in_array($input['user_id'], $userIDsFromDB));

        return $validatedInputs;
    }
}

Let's break down this code

Batchable: Used for handling job batching, making it part of a larger batch of jobs if needed. Make sure to add this trait.
__construct(): When an instance of this job is created, it receives a chunk of CSV data as an array. This data is assigned to the $chunkData property for processing during job execution.
handle(): This method is automatically called by Laravel's queue worker when the job is processed. It's where the main logic of the job resides. In this case, it calls the validation method and then bulk inserts the validated record in the page_analytics table.
validateEachRowAndDiscard(): This function iterates through each row in the chunk, validating it against a set of rules. If a row fails validation, it's skipped. We also want to make sure that the user_id exists in the users table. Now running 500 queries (1 for each row) to check the user_id won't be efficient. We will take all the user_ids using array_column() -> pass it to array_unique() so that we have distinct ids -> make 1 single database call to get all the user_ids from the table using the unique_ids and whereIn clause -> use array_filter() to filter out all the validated rows whose user_id doesn't exists in the database.

Testing it out

Let's test through postman and run the worker using



php artisan queue:work

Conclusion

And that's it, we manage to create a Laravel API to import large size of data in much faster and efficient way. Let me know in the comment section any improvements that can be done further in the code to make it more efficient :)

Build Blazing Fast Websites with Next.js ISR and Headless CMS

Aditya — Thu, 28 Dec 2023 16:08:18 +0000

In the fast-paced world of web development, speed is essential. Users expect lightning-fast page loads, and search engines favour quick-loading sites. With Next.js Incremental Static Regeneration (ISR) you can significantly boost your website's performance.

It's also important to have a smooth content management workflow. A headless CMS is a great option, providing flexibility to use any front-end tech stack. In this article, we'll use Garchi CMS, an easy-to-use SaaS headless CMS.

Let's get started 🚀

1. Garchi CMS setup

Garchi CMS separates content from presentation, ideal for Next.js ISR since we can fetch content via API.

To get started we will need a Garchi CMS account and the API key.

Once the account is created successfully, we get a default space with its id just after the # (we will need it for the api call). Select the default space and then on the top right corner you should see the avatar. Click on that then Subscription and API and at the bottom there should be an option to generate an API token.

In my case, I have already created an API token so that's why in the image you see the text Revoke and Create New Token.

We will put this token inside our Next.js .env file in the next step.

2. Setting up the Next.js project

Garchi CMS has its starter kits. So I will use the Next.js one.

npx @adiranids/garchi-starter-kit -k next

Let's install all the packages that are predefined in the package.json of the starter kit.

npm i

Let's paste the API key generated in step 1 in .env file.

GARCHI_API_URL=https://garchi.co.uk/api/v2
GARCHI_API_KEY=YourApiKey

3. Creating components and connecting them with Garchi CMS.

To have a smooth experience, it is best to keep the component file name inside the project the same as the one created on Garchi CMS. It helps in dynamic import. Any component created on Garchi CMS must exist in the project. More info can be found here

For demonstration, I will show one component example but you can create as many as you want :) All our components will be inside the components/garchi folder.

Most of the components I have created are server components.

Brief.jsx

import { Prose } from '../Prose'


export default async function Brief({ title, content, ...props }) {
    return (
        <div className="lg:order-first lg:row-span-2">
            <h1 className="text-4xl font-bold tracking-tight text-zinc-800 dark:text-zinc-100 sm:text-5xl">
                {title}
            </h1>
            <div className="mt-6 space-y-7 text-base text-zinc-600 dark:text-zinc-400">
                <Prose markdown={content} />
            </div>
        </div>
    )
}

Prose.jsx

import clsx from 'clsx'
import { cleanHTML } from '@/lib/domcontent'

export function Prose({ markdown, className, ...props }) {
  return (
    <div dangerouslySetInnerHTML={{ __html: cleanHTML(markdown)}} className={clsx(className, 'prose dark:prose-invert')} {...props} />
  )
}

Garchi CMS passes HTML string in the api response so it is essential to sanitise it to avoid XSS attack. For that purpose I have created a utility function inside lib/domcontent.js

import { JSDOM } from 'jsdom';
import DOMPurify from 'dompurify';


export function cleanHTML(html) {
  const window = new JSDOM('').window;
  const purify = DOMPurify(window);
  return purify.sanitize(html);
}

To make it work we need to install jsdom and dompurify

npm i jsdom dompurify

Adding components in Garchi CMS

All the components that are used on Garchi CMS should be there in the code base.

I will add a Brief component as a section template which I can then use anywhere on my page on Garchi CMS. I have also added same props to it as I have the one in Brief.jsx.

I can then go to the Pages section in Garchi CMS and edit the page to add the Brief component.

4. Server-side functions to fetch data from Garchi CMS.

Choosing the right data fetching strategy is key for web page performance. For frequently updated, dynamic content, Server Side Rendering (SSR) or Static Site Generation (SSG) with incremental updates works well. SSG alone can be used for mostly static pages.

Incremental Static Regeneration (ISR) strikes a balance - it allows fresh content on pages that don't need full SSR. Next.js generates static pages at build time, then automatically updates them incrementally as content changes. This means fast performance without rebuilding the entire site."

The key points are:

SSR/ISR for dynamic content
SSG for static content
ISR balances fresh content without full rebuild

How ISR works in general

Create Static Pages Once: When someone visits a page for the first time, a static version of that page is generated and saved.
Reuse and Update as Needed: This static page is then reused for other visitors, making it super fast to load. This page can also be updated in the background if the content changes, without you having to rebuild the whole site.
Automatic Refreshing: After a set amount of time (which you decide), the server checks if the page needs updating with new content and refreshes the static page if needed.

How to use ISR in Next.js (app router)

With the pages router, we could use getStaticProps with revalidate and getStaticPaths functions (only for dynamic routes) to achieve ISR.

With Next.js app router, there are some strategies we could use for ISR.

Strategy 1: Time-based revalidation

On any page.jsx/tsx file if we add

//value in seconds
export const revalidate = 3600

Next.js will automatically revalidate that page (look for updated content) after a specified interval (in this case 3600 seconds i.e. 1 hour).

Another approach is passing next.revalidate option in the fetch request.

fetch('https://garchi.co.uk/api/items', { 
headers: yourHeaders,
next: { revalidate: 3600 } 
})

Strategy 2: On-demand revalidation

This is more useful when you want to revalidate a page based on user action or certain system actions or based on certain logic.
You can use this feature inside a server action or route handler using the revalidatePath function (on-demand by path) or the revalidateTag function (on-demand by tag).

As I am using time-based revalidation I won't go in-depth for on-demand revalidation. But you can read more over here

Now we have the basic idea of how ISR works in Next.js app
router, let's see the server functions of our code.

As I am using the starter kit, there are base utility functions provided in utils/garchi.ts. It looks something as below

import { GarchiAsset, GarchiPage } from "@/types/garchi"

class GarchiHelper {

    private baseHeaders : {[x:string] : any }
    private GARCHI_URL: string

    constructor() {
        this.baseHeaders = {
            'Content-Type': 'application/json',
            'Accept': 'application/json',
            'Authorization': `Bearer ${process.env.GARCHI_API_KEY}`
        }
        this.GARCHI_URL = process.env.GARCHI_API_URL as string
    }


    async garchiPostRequest(endpoint: string, payload: any){
        const response = await fetch(`${this.GARCHI_URL}/${endpoint}`, {
            method: 'POST',
            headers: this.baseHeaders,
            body: JSON.stringify(payload)
        })
        return await response.json()
    }


    async garchiGetRequest(endpoint: string){
        const response = await fetch(`${this.GARCHI_URL}/${endpoint}`, {
            method: 'GET',
            headers: this.baseHeaders,
        })
        return await response.json()
    }

    async getGarchiPage(spaceUID: string, mode: "draft" | "live" = "draft" ,pageSlug: string = "/") : 
        Promise<GarchiPage>
    {
        const response = await this.garchiPostRequest("page", {
            space_uid: spaceUID,
            mode,
            slug: pageSlug
        })

        return response
    }


    async getGarchiAsset(spaceUID: string, assetName: string) : Promise<GarchiAsset>
    {
        const response = await this.garchiGetRequest(`space/assets/${assetName}?space_uid=${spaceUID}`)
        return response
    }

}

export default new GarchiHelper()

We need to get the page content using Garch CMS API. For that, there is the getGarchiPage function which needs a slug, space uid and mode. This function calls the http POST request helper function garchiPostRequest. All we have to do is modify this garchiPostRequest function and pass next.revalidate option


//prev code

  async garchiPostRequest(endpoint: string, payload: any){
        const response = await fetch(`${this.GARCHI_URL}/${endpoint}`, {
            method: 'POST',
            headers: this.baseHeaders,
            body: JSON.stringify(payload),
            next: {
                revalidate: 15
            }
        })
        return await response.json()
    }


// after code

In the above function, I am making a fetch POST request and having it revalidated every 15 seconds.

4. Bringing it all together

We now have the helper functions and some components to fetch content and render our page.

In the starter kit, there are two main components in the components/garchi folder. Page.tsx and GarchiComponent.tsx

The purpose of these two components is that once we register all our components in Garchi CMS, we can create pages using these components with all the flexibility we want and we don't have to touch the code. The Page component can be used to render any route and GarchiComponent can be used to render any registered component in any order (sequence) dynamically.

Finally, we need a dynamic route segment to capture all routes for dynamic routing. So within app folder we can create [[...slug]]/page.tsx

The double square brackets around ...slug make it possible to catch all the routes including the index/home route.

/app/[[...slug]]/page.tsx

import Page from "@/components/garchi"

type Props =  { params: { slug: string[] } }

export default function DynamicPage({ params }: Props) {
  return <Page slug={params?.slug?.join("/")} />
}

There you have it - a blazing-fast site with headless CMS and ISR!

Headless blog using Garchi CMS & Nuxt 3 - Part 1

Aditya — Wed, 19 Jul 2023 08:24:48 +0000

Nuxt 3 is an incredibly amazing Vue.js meta-framework. Its exceptional performance, delightful developer experience, and plethora of features make it a popular choice among developers. In this post, we'll explore how we can harness Nuxt 3's capabilities in conjunction with Garchi CMS, a powerful headless CMS, to create a headless blog.

Let's first demystify a couple of key terms before diving in.

1. Headless CMS

A headless CMS is a type of content management system that provides back-end content management functionalities without any predefined front-end delivery layer. It disseminates content through APIs to any platform or device, empowering developers to construct custom front-ends. This method ensures greater flexibility, scalability, and swifter content delivery.

2. Garchi CMS

Garchi CMS is our proprietary, cloud-based (SaaS) headless CMS. Initially built for the e-commerce industry, it has since been extended to accommodate similar systems like e-learning platforms, blogs, and more. Its core appeal lies in its minimalism and the inclusion of REST APIs. No need to install packages or perform any initial setup other than generating an API token. Its lightweight nature allows it to be used as a component in a diverse range of applications or software.

Tech stacks we will use

Nuxt 3
Tailwind CSS
Garchi CMS

Now that we're up to speed, let's configure our blog's backend on Garchi CMS. For simplicity, we won't be including authentication, but feel free to integrate it with services like Firebase, Supabase, or any other of your choosing.

Start by visiting Garchi CMS website and registering a free account.

Upon successful registration, you'll land on a dashboard with a default space.

'Space' in Garchi CMS can be thought of as a workspace dedicated to a single system. For example, one space might be reserved for blog articles, while another could cater to product listings. You can either continue with the default space or create a new one.

I will go with the default space and we should see a dashboard like this

We'll begin by establishing categories. You can find this option in the sidebar, under 'Data Set -> Manage Categories'. Add a new category, for example, 'Blog'.

This will be our parent category for blogs and within that we can have sub categories. So I will create another category say Web Dev.

With our categories established, we can proceed to add items. Garchi CMS's item schema echoes its e-commerce roots, but you can extend it with additional features or data points using the 'Add extra details' function.

To create a new item (which would be a blog post in our case), click on 'Add New Item'.

For our example, let's generate a simple blog on Vue 3, classified under the 'Blog' and 'Web Dev' categories. To expedite the process, we'll use Garchi's built-in GPT support to generate an item description (which will be our blog article body). Although our example utilizes text content, you can include images, videos, and embeds.

(Don't worry this current article is all written by me :D)

We can keep the selling price to 0.

So we would have something like this

Once you've added the necessary details, you can either 'Save and Publish' immediately or schedule it for a later time.

You can augment your post with additional information using the 'Add extra details' option under the item. Let's add details like the author's name, publication date, and avatar.

These additional details are formatted as key-value pairs, and you'll need to save each section individually. These pairs will be useful to extract metadata or filter items using the Garchi CMS API later.

Next, add an image for the blog post by clicking on the image icon.

Garchi CMS allows you to add up to four images per post. However, to keep things simple, we'll only add one image for now.

Finally, generate an API key to use Garchi CMS APIs from the Nuxt 3 end. Click on 'API' in the sidebar and then 'Create New Token'.

You should now see your API key. Keep this safe, as it's essential for billing purposes. You can also invite more team members to collaborate on your project.

That's it for this part. We'll continue setting up the Nuxt 3 end in our next part.

Headless blog using Garchi CMS & Nuxt 3 - Part 2

Aditya — Wed, 19 Jul 2023 08:24:34 +0000

In part 1, we set up our blog's backend with the Garchi CMS. Let's now move forward to constructing the frontend using Nuxt 3.

To begin, ensure the latest version of Node JS is installed on your system.

In your terminal, execute the following command to create a new Nuxt 3 application. You're free to replace 'blog' with any other name that you prefer.

npx nuxi@latest init blog

After execution, you should see an output like the following:

Now, navigate into the 'blog' folder and run npm i to install the necessary dependencies.

cd blog
npm i

To expedite the process, I'll bypass the fundamentals of Nuxt 3 and Tailwind. You can find details on integrating Tailwind with Nuxt 3 here

Next, incorporate the API key created in the part 1 within our nuxt.config.ts file and add the API base URL as shown in the image.

We'll swap NuxtWelcome with NuxtPage in app.vue and create index.vue in the 'pages' directory.

Remember, it's advisable to employ Garchi API server-side due to the API key's billing purposes. Hence, let's create an API route in Nuxt 3 by generating a file named 'blog.get.ts' which will map to the /api/blog GET request. This file should be located within the 'server/api' directory.

Include the following code within 'blog.get.ts':

export default defineEventHandler(async (event) => {
    const categoryId = 71
    const config = useRuntimeConfig()

    const response = await $fetch(`${config.API_URL}/products/filter`, {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'Authorization': `Bearer ${config.API_KEY}`
        },
        body: {
            categories: [categoryId]
        }
    })

    return response
})

Let's dissect the code:

We will utilize Garchi CMS's filter products API, which can be found here This API accepts a list of category ids, space ids, or priceOrder (for e-commerce) and filters based on that. From our Garchi CMS dashboard, we know the category id and will use that. We'll pass it as an array since the API accepts an array of ids. Consequently, this should return all the items categorised as 'blog' since 71 (in my case) is the id of the 'blog' category.
We make a POST request to https://garchi.co.uk/api/v1/products/filter and for now, we'll merely return the response to examine the output.

As this is a GET API from the Nuxt end, we can execute it in the browser to view the results. Hence, we'll start the dev server with:

npm run dev

We'll employ this information to display our list of blog posts. Therefore, I'll construct a component within the 'components' folder and label it 'BlogCard.vue'. But first, let's create types based on the API response we received. So within 'assets/types/Post.ts' include this code:


export type PostMeta = {
    id: number
    key: string
    value: string
    type: string
}

export type Post = {
    product_id: number
    slug: string
    name: string
    description?: string
    categories: {
        id: number
        name: string
    }[]
    main_image: string
    space: {
        uid: string
        name: string
    }
    product_meta: PostMeta[]
}

The code in our 'BlogCard.vue' should look like this:

<template>
  <article class="flex flex-col items-start justify-between">
    <div class="relative w-full">
      <img
        :src="post.main_image"
        alt=""
        class="aspect-[16/9] w-full rounded-2xl bg-gray-100 object-cover sm:aspect-[2/1] lg:aspect-[3/2]"
      />
      <div
        class="absolute inset-0 rounded-2xl ring-1 ring-inset ring-gray-900/10"
      />
    </div>
    <div class="max-w-xl">
      <div class="mt-8 flex items-center gap-x-4 text-xs">
        <NuxtLink
          class="relative z-10 rounded-full bg-gray-50 px-3 py-1.5 font-medium text-gray-600 hover:bg-gray-100"
        >
          {{ categoryTitle }}
        </NuxtLink>
      </div>
      <div class="group relative">
        <h3
          class="mt-3 text-lg font-semibold leading-6 text-gray-900 group-hover:text-gray-600"
        >
          <NuxtLink :to="`/blog/${post.slug}`">
            <span class="absolute inset-0" />
            {{ post.name }}
          </NuxtLink>
        </h3>
      </div>
      <div class="relative mt-8 flex items-center gap-x-4">
        <img
          :src="authorImage"
          alt=""
          class="h-10 w-10 rounded-full bg-gray-100"
        />
        <div class="text-sm leading-6">
          <p class="font-semibold text-gray-900">
            <span>
              <span class="absolute inset-0" />
              {{ authorName }}
            </span>
          </p>
        </div>
      </div>
    </div>
  </article>
</template>

<script setup lang="ts">
import { Post, PostMeta } from "~/assets/types/Post";

const postProps = defineProps<{
  post: Post;
}>();

const categoryTitle = computed(() =>
  postProps.post?.categories.map((category) => category.name).join(", ")
);

const authorImage = computed(
  () =>
    postProps.post?.product_meta?.find((m) => m.key == "avatar")?.value ?? ""
)
const authorName = computed(
  () =>
    postProps.post?.product_meta?.find((m) => m.key == "author")?.value ?? ""
)
</script>

If you notice, I have taken authorImage, authorName and categoryTitle as computed properties. This is because authorImage and authorName will be inside the meta detail (product_meta) as we added these values as extra fields. Category will be an array of categories so we will join it using ', '.

Next inside our index.vue we will have something like below

<template>
    <Head>
        <Title>Blog</Title>
    </Head>
    <div class="max-w-6xl p-2 mx-auto xl:py-20
    grid grid-cols-1 md:grid-cols-2 xl:grid-cols-3 gap-4 md:gap-5">
        <BlogCard v-for="(post, index) in data?.data" :key="index" :post="post" />
    </div>
</template>

<script setup lang="ts">
const {data} = await useFetch("/api/blog")
</script>

Next, we will create a dynamic route which will render our blog article on click. So inside pages/blog create [slug].vue file. You can also create a layout to have less code rewrite. But for this example, I will rewrite the classes, sorry about that :)

Also a layout can be created so that we have less code rewrite. But for this example I will rewrite the classes. Sorry about that :)

Previously, we created an API route. This time we could utilise Nuxt 3's useAsyncData with the server key set to true, so it runs on the server. We will use Get Product Details of Garchi CMS.

Since the API returns HTML in the description, we need to clean it using the DOMPurify package. We can also allow only iframe tags for YouTube so let's do that.

First, install dompurify:

npm i dompurify

If you are using TypeScript, you might need to add the types:

npm i --save-dev @types/dompurify

Our [slug].vue should look something like this:

<template>
  <Head>
    <Title>{{ article?.name }}</Title>
  </Head>
  <div class="max-w-6xl p-2 mx-auto xl:py-20 flex flex-col gap-9">
    <img
      :src="article?.main_image"
      alt=""
      class="aspect-square w-full xl:w-[30%] xl:h-1/2 mx-auto rounded-2xl bg-gray-100 object-cover object-center"
    />

    <h1 class="text-4xl">
      {{ article?.name }}
    </h1>

    <div class="flex items-center space-x-3">
      <img
        :src="authorImage"
        :alt="authorName"
        class="w-16 h-16 rounded-full ring-2 ring-gray-900 ring-offset-gray-800"
      />
      <span class="text-gray-600 font-semibold text-lg">
        By {{ authorName }}
      </span>
    </div>

    <div
      class="prose max-w-full"
      v-html="description || article?.description"
    ></div>
  </div>
</template>

<script setup lang="ts">
import { Post } from "assets/types/Post";
import * as DOMPurify from "dompurify";

const config = useRuntimeConfig();
const route = useRoute();

const { data, error } = await useAsyncData<{ data: Post[] }>(
  `article-${route.params.slug}`,
  () => {
    return $fetch(`${config.API_URL}/product/${route.params.slug}`, {
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${config.API_KEY}`,
      },
    });
  },
  {
    server: true,
  }
);

const article = computed<Post | null>(() => data.value?.data[0] ?? null);

const description = ref<string>("");

const authorImage = computed(
  () => article.value?.product_meta?.find((m) => m.key == "avatar")?.value ?? ""
);
const authorName = computed(
  () => article.value?.product_meta?.find((m) => m.key == "author")?.value ?? ""
);

if (error.value || !article.value) {
  throw createError({ statusCode: 404, message: "Not found" });
}

onMounted(() => {
  DOMPurify.addHook("uponSanitizeElement", (node: HTMLElement, data: any) => {
    if (data.tagName == "iframe") {
      const allowedSRC = "https://www.youtube.com/embed";
      const src = node.getAttribute("src");
      if (!src?.startsWith(allowedSRC)) {
        return node.parentNode?.removeChild(node);
      }
    }
  });

  let content = DOMPurify.sanitize(article.value?.description, {
    USE_PROFILES: { html: true },
    ADD_TAGS: ["iframe"],
    ADD_ATTR: ["autoplay", "allowfullscreen", "frameborder", "scrolling"],
  });

  description.value = content;
});
</script>

Let's break down the code step by step:

The API returns data in the format {data: Post[]}

We have a computed property which gives us data:Post[0]. This is because the API, which accepts either a slug or an item's id as a URL parameter, will return only one item or none in the response array.

We are using DOMPurify to sanitize our article.description which is in HTML format and allow only certain iframes. As this is going to happen on the client side because of the onMounted hook, I am showing article.description that comes from the server and switch it with client side description when it is sanitized. This can be handled completely on the server side, I guess.

Finally, we spice it up with the Tailwind Typography plugin.
The final output is:

To conclude, the main intention of this two-part article is to demonstrate the usage of Garchi CMS and a quick demo of how it could be used.

That's all for now, hope you enjoyed it :)

Atomic Design in Laravel using blade components

Aditya — Thu, 08 Jul 2021 16:52:18 +0000

Structuring your front end code can become cumbersome as your project starts to increase. Atomic design thinking is an amazing way to structure your code and achieve flexibility, consistency and code reuse.

Laravel blade components is an amazing way to create server side rendered components. Though they are not like React or Vue components but they are bit similar in the way that you can pass the props and have the code reused. In this video, I will show how to do atomic design in a Laravel app using blade components.

https://youtu.be/XvqMPsdEiSQ