DEV Community: Christonit

AWS + JavaScript + WordPress = Fun Content Automation Strategies Using Artificial Intelligence

Christonit — Sat, 04 Jan 2025 03:53:49 +0000

Months ago, I began collaborating on a project all about AI generated content for a client focused on the tech sector. My role was mostly focused on setting up SSG using WordPress as a Headless CMS for a Nuxt Front-end.

The client used to write articles a couple of times per week about different trends or situations affecting the sector, in the hope of increasing traffic to the site and its output of articles he decided to use AI to generate articles for him.

After some time, with the right prompts the client had pieces of information that were close to an exact match of a human written article, it is super difficult to spot they are machine made.

Sometime after I moved to work on different features, I would continuously get asked one specific thing.

Ey, can you update the featured image for this article?

After 2 weeks of daily updating Posts I had a small eureka moment.

Why don't I automate the featured image generation for these articles using Artifical Intelligence?

We already automated post writing, why don't automate the featured images?

In my free time, I was experimenting with generative LLMs on my computer so I had a solid idea of more or less how to tackle this side-quest. I sent a message to the client detailing what is the problem, what I want to do and what were going to be the advantages and without having to do convincing, I got the green-lit to work on this feature and right away I went with my first step.

1. Architecting how the solution is going to look.

Given that I had some exposure to running models locally I knew right away it was not feasible to self host those models. With that discarded I started to play around APIs that generated images based on text prompts.

Featured images consisted of 2 parts: the main composed graphic and a catchy tagline.

The composed graphic would be some elements related to the article, arranged in a nice way with then some colors and textures with some blend modes applied to achieve some fancy effects following the branding.

Taglines were short, 8-12 words sentences with a simple drop shadow under them.

Based on my testing, I realized that pursuing the AI route for image generation wasn’t practical. The image quality didn’t meet expectations, and the process was too time-consuming to justify its use. Considering this would run as an AWS Lambda function, where execution time directly impacts costs.

With that discarded, I went with Plan B: mashing images and design assets together using JavaScript's Canvas API.

Taking a deep look we had mainly 5 styles of simple posts, and around 4 types of textures and 3 of them using the same text alignment, style and position. After doing some math I thought:

Hmm, If i take these 3 images, grab 8 textures and play with blend-modes, I can get around post 24 variations

Given that those 3 types of posts had the same text style it was practically one template.

With that settled, I moved to the Tagline Generator. I wanted to create a tagline based on the content and title of the article. I decided to use ChatGPT’s API given that the company was already paying for it, and after some experimenting and promps tweaking, I had a very good MVP for my tagline generator.

With the 2 hardest parts of the task figured out, I spent some time in Figma putting together the diagram for the final architecture of my service.

2.Coding my lambda

The plan was to create a Lambda function capable of analyzing post content, generating a tagline, and assembling a featured image—all seamlessly integrated with WordPress.

I will provide some code but just enough to communicate the overall idea to ke.

Analyzing the content

The Lambda function starts by extracting the necessary parameters from the incoming event payload:
const { title: request_title, content, backend, app_password} = JSON.parse(event.body);

title and content: These provide the article’s context.
backend: The WordPress backend URL for image uploads.
app_password: The authentication token im going to use to upload as my user using Wordpress Rest API.

Generating the Tagline

The function’s first major task is to generate a tagline using analyzeContent function, which uses OpenAI’s API to craft a click-worthy tagline based on the article's title and content.

Our function takes the post title and content but returns a tagline, post sentiment to know if post is a positive, negative or neutral opinion and an optional company symbol from the S&P index companies.

const { tagline, sentiment, company } = await analyzeContent({ title: request_title, content });

This step is critical, as the tagline directly influences the image’s aesthetics.

Creating the Featured Image

Next, the generateImage function kicks in:

let buffer;

buffer = await generateImage({
    title: tagline,
    company_logo: company_logo,
    sentiment: sentiment,
});

This function handles:

Designing the composition.
Layering textures, colors, and branding elements.
Applying effects and creating the title.

Here is a step by step breakdown on how it works:

The generateImage function begins by setting up a blank canvas, defining its dimensions, and preparing it to handle all the design elements.

const COLOURS = {
        BLUE: "#33b8e1",
        BLACK: "#000000",
    }

    const __filename = fileURLToPath(import.meta.url);
    const __dirname = path.dirname(__filename);
    const images_path = path.join(__dirname, 'images/');
    const files_length = fs.readdirSync(images_path).length;
    const images_folder = process.env.ENVIRONMENT === "local"
        ? "./images/" : "/var/task/images/";


    registerFont("/var/task/fonts/open-sans.bold.ttf", { family: "OpenSansBold" });
    registerFont("/var/task/fonts/open-sans.regular.ttf", { family: "OpenSans" });


    console.log("1. Created canvas");

    const canvas = createCanvas(1118, 806);

    let image = await loadImage(`${images_folder}/${Math.floor(Math.random() * (files_length - 1 + 1)) + 1}.jpg`);


    let textBlockHeight = 0;

    console.log("2. Image loaded");

    const canvasWidth = canvas.width;
    const canvasHeight = canvas.height;
    const aspectRatio = image.width / image.height;


    console.log("3. Defined ASPECT RATIO",)

    let drawWidth, drawHeight;
    if (image.width > image.height) {
        // Landscape orientation: fit by width
        drawWidth = canvasWidth;
        drawHeight = canvasWidth / aspectRatio;
    } else {
        // Portrait orientation: fit by height
        drawHeight = canvasHeight;
        drawWidth = canvasHeight * aspectRatio;
    }

    // Center the image
    const x = (canvasWidth - drawWidth) / 2;
    const y = (canvasHeight - drawHeight) / 2;
    const ctx = canvas.getContext("2d");
    console.log("4. Centered Image")
    ctx.drawImage(image, x, y, drawWidth, drawHeight);

From there, a random background image is loaded from a predefined collection of assets. These images were curated to suit the tech-oriented branding while allowing for enough variety across posts. Background image is selected randomly based on its sentiment.

To ensure each background image looked great, I calculated its dimensions dynamically based on the aspect ratio. This avoids distortions while keeping the visual balance intact.

Adding the Tagline

The tagline is short but based on some rules, this impactful sentence is split into manageable pieces and is styled dynamically to ensure it’s always readable, regardless of length or canvas size based on the word count for the line, word length, etc.

console.log("4.1 Text splitting");
if (splitText.length === 1) {

    const isItWiderThanHalf = ctx.measureText(splitText[0]).width > ((canvasWidth / 2) + 160);
    const wordCount = splitText[0].split(" ").length;

    if (isItWiderThanHalf && wordCount > 4) {

        const refactored_line = splitText[0].split(" ").reduce((acc, curr, i) => {
            if (i % 3 === 0) {
                acc.push([curr]);
            } else {
                acc[acc.length - 1].push(curr);
            }
            return acc;
        }, []).map((item) => item.join(" "));

        refactored_line[1] = "[s]" + refactored_line[1] + "[s]";

        splitText = refactored_line

    }
}

let tagline = splitText.filter(item => item !== '' && item !== '[br]' && item !== '[s]' && item !== '[/s]' && item !== '[s]');
let headlineSentences = [];
let lineCounter = {
    total: 0,
    reduced_line_counter: 0,
    reduced_lines_indexes: []
}

console.log("4.2 Tagline Preparation", tagline);

for (let i = 0; i < tagline.length; i++) {
    let line = tagline[i];

    if (line.includes("[s]") || line.includes("[/s]")) {

        const finalLine = line.split(/(\[s\]|\[\/s\])/).filter(item => item !== '' && item !== '[s]' && item !== '[/s]');

        const lineWidth = ctx.measureText(finalLine[0]).width
        const halfOfWidth = canvasWidth / 2;

        if (lineWidth > halfOfWidth && finalLine[0]) {

            let splitted_text = finalLine[0].split(" ").reduce((acc, curr, i) => {

                const modulus = finalLine[0].split(" ").length >= 5 ? 3 : 2;
                if (i % modulus === 0) {
                    acc.push([curr]);
                } else {
                    acc[acc.length - 1].push(curr);
                }
                return acc;
            }, []);

            let splitted_text_arr = []

            splitted_text.forEach((item, _) => {
                let lineText = item.join(" ");

                item = lineText

                splitted_text_arr.push(item)
            })

            headlineSentences[i] = splitted_text_arr[0] + '/s/'

            if (splitted_text_arr[1]) {
                headlineSentences.splice(i + 1, 0, splitted_text_arr[1] + '/s/')
            }
        } else {
            headlineSentences.push("/s/" + finalLine[0] + "/s/")
        }


    } else {
        headlineSentences.push(line)
    }
}

console.log("5. Drawing text on canvas", headlineSentences);

const headlineSentencesLength = headlineSentences.length;
let textHeightAccumulator = 0;

for (let i = 0; i < headlineSentencesLength; i++) {
    headlineSentences = headlineSentences.filter(item => item !== '/s/');
    const nextLine = headlineSentences[i + 1];
    if (nextLine && /^\s*$/.test(nextLine)) {
        headlineSentences.splice(i + 1, 1);
    }

    let line = headlineSentences[i];

    if (!line) continue;
    let lineText = line.trim();

    let textY;

    ctx.font = " 72px OpenSans";

    const cleanedUpLine = lineText.includes('/s/') ? lineText.replace(/\s+/g, ' ') : lineText;
    const lineWidth = ctx.measureText(cleanedUpLine).width
    const halfOfWidth = canvasWidth / 2;

    lineCounter.total += 1

    const isLineTooLong = lineWidth > (halfOfWidth + 50);

    if (isLineTooLong) {

        if (lineText.includes(':')) {
            const split_line_arr = lineText.split(":")
            if (split_line_arr.length > 1) {
                lineText = split_line_arr[0] + ":";
                if (split_line_arr[1]) {
                    headlineSentences.splice(i + 1, 0, split_line_arr[1])
                }
            }
        }

        ctx.font = "52px OpenSans";

        lineCounter.reduced_line_counter += 1

        if (i === 0 && headlineSentencesLength === 2) {
            is2LinesAndPreviewsWasReduced = true
        }


        lineCounter.reduced_lines_indexes.push(i)

    } else {

        if (i === 0 && headlineSentencesLength === 2) {
            is2LinesAndPreviewsWasReduced = false
        }


    }

    if (lineText.includes("/s/")) {

        lineText = lineText.replace(/\/s\//g, "");

        if (headlineSentencesLength > (i + 1) && i < headlineSentencesLength - 1 && nextLine) {

            if (nextLine.slice(0, 2).includes("?") && nextLine.length < 3) {
                lineText += '?';
                headlineSentences.pop();
            }

            if (nextLine.slice(0, 2).includes(":")) {
                lineText += ':';
                headlineSentences[i + 1] = headlineSentences[i + 1].slice(2);
            }

        }

        let lineWidth = ctx.measureText(lineText).width


        let assignedSize;


        if (lineText.split(" ").length <= 2) {

            if (lineWidth > (canvasWidth / 2.35)) {

                ctx.font = "84px OpenSansBold";

                assignedSize = 80

            } else {

                ctx.font = "84px OpenSansBold";

                assignedSize = 84

            }
        } else {


            if (i === headlineSentencesLength - 1 && lineWidth < (canvasWidth / 2.5) && lineText.split(" ").length === 3) {

                ctx.font = "84px OpenSansBold";
                assignedSize = 84

            } else {

                lineCounter.reduced_line_counter += 1;

                ctx.font = "52px OpenSansBold";
                assignedSize = 52

            }

            lineCounter.reduced_lines_indexes.push(i)

        }

        lineWidth = ctx.measureText(lineText).width



        if (lineWidth > (canvasWidth / 2) + 120) {

            if (assignedSize === 84) {
                ctx.font = "72px OpenSansBold";
            } else if (assignedSize === 80) {
                ctx.font = "64px OpenSansBold";

                textHeightAccumulator += 8
            } else {
                ctx.font = "52px OpenSansBold";
            }
        }



    } else {

        const textWidth = ctx.measureText(lineText).width


        if (textWidth > (canvasWidth / 2)) {
            ctx.font = "44px OpenSans";
            textHeightAccumulator += 12
        } else if (i === headlineSentencesLength - 1) {
            textHeightAccumulator += 12
        }

    }

    ctx.fillStyle = "white";
    ctx.textAlign = "center";

    const textHeight = ctx.measureText(lineText).emHeightAscent;

    textHeightAccumulator += textHeight;

    if (headlineSentencesLength == 3) {
        textY = (canvasHeight / 3)
    } else if (headlineSentencesLength == 4) {
        textY = (canvasHeight / 3.5)
    } else {
        textY = 300
    }

    textY += textHeightAccumulator;

    const words = lineText.split(' ');
    console.log("words", words, lineText, headlineSentences)
    const capitalizedWords = words.map(word => {
        if (word.length > 0) return word[0].toUpperCase() + word.slice(1)
        return word
    });
    const capitalizedLineText = capitalizedWords.join(' ');

    ctx.fillText(capitalizedLineText, canvasWidth / 2, textY);

}

Finally, the canvas is converted into a PNG buffer.

const buffer = canvas.toBuffer("image/png");
return buffer;

Finally!!! Uploading the Image to WordPress

After successfully generating the image buffer, the uploadImageToWordpress function is called.

This function handles the heavy lifting of sending the image to WordPress using its REST API by Encoding the Image for WordPress.

The function first prepares the tagline for use as the filename by cleaning up spaces and special characters:

const createSlug = (string) => {
    return string.toLowerCase().replace(/ /g, '-').replace(/[^\w-]+/g, '');
};

const image_name = createSlug(tagline);

The image buffer is then converted into a Blob object to make it compatible with the WordPress API:

const file = new Blob([buffer], { type: "image/png" });

Preparing the API Request Using the encoded image and tagline, the function builds a FormData object and I add optional metadata, such as alt_text for accessibility and a caption for context.

formData.append("file", file, image_name + ".png");
formData.append("alt_text", `${tagline} image`);
formData.append("caption", "Uploaded via API");

For authentication, the username and application password are encoded in Base64 and included in the request headers:


const credentials = `${username}:${app_password}`;
const base64Encoded = Buffer.from(credentials).toString("base64");

Sending the Image A POST request is made to the WordPress media endpoint with the prepared data and headers and after awaiting the response I validate for success or errors.

const response = await fetch(`${wordpress_url}wp-json/wp/v2/media`, {
    method: "POST",
    headers: {
        Authorization: "Basic " + base64Encoded,
        contentType: "multipart/form-data",
    },
    body: formData,
});

if (!response.ok) {
    const errorText = await response.text();
    throw new Error(`Error uploading image: ${response.statusText}, Details: ${errorText}`);
}

If successful, I return that same media response in the lambda.

This is how my lambda looks in the end.

import { analyzeContent } from './ai-analysis.js';
import { uploadImageToWordpress, generateImage } from './draw.js';

export const handler = async (event) => {
    try {
        const { title: request_title, content, backend, app_password, return_image } = JSON.parse(event.body);

        if (typeof request_title !== 'string' || typeof content !== 'string') {
            return {
                statusCode: 400,
                body: JSON.stringify({ error: 'Invalid input' }),
            };
        }

        const { tagline, company_logo, sentiment } = await analyzeContent({ title: request_title, content });

        const BACKENDS = ["https://content.x.com/", "https://content.y.com/", "https://content.z.com/"]

        let buffer;

        console.log("1. SUCCESSFUL Tagline Creation", { tagline, sentiment, backend }, BACKENDS.includes(backend));


        buffer = await generateImage({
            title: tagline,
            company_logo: company_logo,
            sentiment: sentiment,
        });

        console.log("3. SUCCESSFUL Image Creation for", backend);
        if (buffer) {

            const response = await uploadImageToWordpress(backend, buffer, tagline, app_password);

            if (response.status !== 201) {

                throw new Error("Image Upload Failed" + response.statusText);
            }

            const data = await response.json();

            console.log("4. SUCCESSFUL Image Upload for", backend);
            return {
                statusCode: 200,
                headers: {
                    'Content-Type': 'application/json',
                },
                body: data,
            };

        }
    } catch (error) {
        return {
            statusCode: 500,
            body: JSON.stringify({ error: error.message }),
        };
    }
};

This is a sample image produced by my script. It's not used in production, just created with generic assets for this example.

Aftermath

Some time has passed and everybody is happy that we no longer have shoddy or empty looking image-less articles, that images are a close match to the ones that the designer crafts, the designer is happy that he gets to only focus on designing for other marketing efforts across the company.

But then a new problem arose: sometimes the client did not like the Image generated and he would ask me to spin up my script to generate a new one for a specific post.

This brought me to my next sidequest: A Wordpress Plugin to Manually Generate a Featured image using Artificial Inteligence for an Specific Post

Case Study: Migrating a high-traffic WordPress website to a Headless Architecture using Nuxt

Christonit — Fri, 16 Feb 2024 18:29:25 +0000

In this case study, I'll share a small sneak peek of my experience of migrating a traditional WordPress site to a redesigned front-end and high-performance headless architecture using Nuxt.

I’ll highlight our challenges, focus on what we think to be the most important and pivotal moments of our decision-making process, and the ultimate positive impact our move to Nuxt made on our lead generation website.

A little bit of history

I have been working for my client for almost 4 years, for the first 2 years all I was doing was marketing microsites and small experiments but that changed in the second quarter of 2022.

A highly visited website was around 12 years old and with the last redesign circa 2015, the whole website was showing its age. We use WordPress and a custom theme.

The problems we had in 2022 were the same problems everybody running a business with WordPress might encounter:

Technical Debt in the Form of Plugin Updates: Our website was burdened by outdated plugins. which threatened to break our site with every attempt at an update.
Oversized Network Payload: probably again because of out-of-use plugins that had to inject many dependencies into our website
Unnecessary Markup
Slow Website Load Times (sometimes)

We did try to solve the technical debt situation. Despite our best efforts to optimize the site, even disabling all tracking scripts in incognito mode to get a cleaner performance read, many pages stubbornly remained in the low 70s according to Lighthouse scores. For those unfamiliar, Lighthouse is a tool by Google that measures website quality across several metrics, including performance. Scores in the 70s indicate significant room for improvement, especially in terms of speed and user experience.

Around April that’s when the Tech Director said “Ey, why don’t we go headless?”.

I will try to be as faithful to the story as possible but the story (more or less) was this way.

Choosing the Tech Stack.

We were a small team back then:

A front-end developer
Me. Another front-end developer with back then around 6 years of experience and a couple of years of experience using React and VueJs.

3 devs counting my boss.

After the project started we were joined by an SEO specialist that had experience in these kinds of projects. He played a crucial role in providing UX quality assurance and in maintaining and reviewing the integrity of the information so SEO-wise everything ended up as close as possible to the back-then live site.

We had never done anything similar with WordPress before: A website using WordPress as a headless CMS?

I had done some freelancing using React where I used Contentul, Strapi, or Builder.io to control the text, toggle components, site-wide announcements, and navigation for SPAs but not with an SEO, SSR, or SSG focus but anyway pitched the idea.

Well, why don't we go with NextJs or Gatsby? I have plenty of experience with React and many articles cover what we are trying to achieve.

The conversation went somewhat like this:

Boss: Maybe later, let’s stick with VueJs. All 3 of us know it and there are others in the company that know it in case we need people to chime in.

Me: Alright, I guess we are going with Nuxt then.

Boss: No, we are going to use Gridsome, we are taking the Static Site Generation route.

The Short-lived Gridsome Era

I was excited. I had been reading about SSG for the past 12 months and was eager for a new challenge. The prospect of serving our pages with less latency and the best performance for the client and achieving above the 80s in the lighthouse sounded promising.

When we researched about it, we found that it was a favorite tool in the Vue community for headless implementation because of how well it interacted with WordPress

It had most of what was provided by a modern front-end framework plus nice interfaces to generate dynamic pages/templates from WordPress.

It generated the graphQL data from WordPress without having to install plugins in our backend. Had good documentation on how to do middleware or load custom data when the server started.

It did not matter however how much we liked Gridsome, reality soon set in with two major hurdles:

Maintenance and Popularity: The last update to Gridsome was in November 2020, leaving it behind with Vue 2's limitations, lacking the Composition API and Vite support. Its niche adoption meant solutions to our challenges were scarce.

Data Handling: Our decade-old site, boasting over published 9,000 posts, strained Gridsome's capabilities. Server start times and build processes became painstakingly long, with frequent crashes due to the overwhelming demand on our WordPress backend.

Long story short, we began development around mid-April, and after shedding blood and tears, and facing the hurdles of the typical learning curve and the technical issues described above plus many more (that could make its list called “The Headless Wordpress with VueJS Cheat sheet”) we launched in September of 2022.

A couple of months after we went live and even after many optimizations, just starting the server could take from 5 to 10 minutes. By the time we went live with Gridsome, our build times ended up being 4 to 6 minutes long after much optimization and content pruning a couple of months after launching.

That meant that on every build, we generated almost 9k pages. And every page/post update on our WordPress backend triggered a new build. And we had to run those builds to show the latest posts or content updates. More than once our builds would crash because our (now) WordPress backend couldn’t handle that many calls and requests back to back.

You might ask “Why are you talking about WordPress to Gridsome if in the title you refer to ‘Moving an old high-traffic WordPress website to Nuxt’?”

Because the first part of the tale on how the coconut gets its water: was the gruesome rebuild we did.

All the lessons learned there, the limitations, and the issues became the foundations and gave enough experience & proof to convince everybody that Nuxt is the way to go for the next batch of websites.

Yes, this was supposed to be the first of many websites to transition to SSG, it was the prototype to benchmark to create the template on how to work the next step: the website’s redesign.

Spoiler alert, the other websites were migrated to Nuxt.

Redesigning a WordPress Website using Nuxt

Almost one year later after the Gridsome launch, we got the task of doing the facelift.

Only this time, nobody was against using Nuxt.

The first step we took was to list all the non-dynamic pages we were going to use and what was going to be dynamic.

The second step was installing the WPGraphQL plugin, which allows us to fetch data using GraphQL.

This plugin allows us to create and test the queries visually without having lots of GraphQL knowledge.

The data our website uses is rendered server side so we prepped our fetch function that we later are going to use on useAsyncData. This will prove beneficial later because we ended up reducing the Layout Content Shift significantly for our website, all the data that our above-the-fold components need is already available on mount.

We have different page types that need different information but a couple of them do share queries. That’s why I moved the small part of the queries to its own utility const file.

The main benefit of using GraphQL is efficiency: We can retrieve all the data required by your Nuxt.js blog in a single request, unlike REST, where we need to make multiple requests to different endpoints to fetch the same data.

Spoiler alert: This *efficiency benefit would become our bane, the core of a show-stopper a couple of weeks after our successful launch.

A good idea is to make a Postman collection with all the API calls you need to do.

After a couple of days of work, we had wired up most of the pages and components for our blog. All that was left was to make public our sitemap for Google to crawl once the page went live and replace inside the post content all the references to internal pages that point to our backend to our current hostname.

<section
  ref="content"
  class="content"
  v-html="contentFormatter(post.content.rendered)"
/>

Remember that we are using Nuxt as a front-end layer, we are going to end up pointing our host to our production domain and then make the WordPress instance point to a different domain.

For this story, let's say our backend is going to end up called backend.wordpress.com.

For replacing the links, we used Cheerio. Cheerio allowed us to process our content as Nodelists and loop through our content and its tags.

const contentFormatter = (content) => {
  const $ = load(content, {
    decodeEntities: false,
    xmlMode: false
  });

  $("a").each((_index, el) => {
    if (
      el.attribs.href &&
      el.attribs.href.includes($config.public.current_hostname)
    ) {
      el.attribs.href = el.attribs.href.replace(
        $config.public.current_hostname,
        $config.public.current_hostname.slice(0, -1)
      );
    }
  });

  const regex = new RegExp(
    "\\b" + $config.public.hostname + "(?!\\.(jpg|jpeg|png|webp|pdf|gif))\\b",
    "gi"
  );

  if (el.attribs.href && el.attribs.href.match(regex)) {
    const extracted_slug = el.attribs.href.split(".com/")[1];
    $(el).attr("href", $config.public.current_hostname + extracted_slug);
  }

For the sitemap, we did not find a plugin that allowed us to produce or directly download the ones provisioned by WordPress, and even if we found one, we still would have to change the references to posts using our backend URL.

For this issue, we wrote a simple script that runs before our Nuxt build that fetches the main sitemap from WordPress and then loops its content to download more sitemaps. Will all the files in hand, I loop once again to replace our references to our backend for the current hostname. The current hostname would be the URL of where we deployed the website.

// Import required libraries
import axios from "axios";
import fs from "fs";
import path from "path";
import xml2js from "xml2js";
import dotenv from "dotenv";

dotenv.config();
const __filename = new URL(import.meta.url).pathname;
const __dirname = path.dirname(__filename);
const targetURL = process.env.HOSTNAME;
const current_hostname = process.env.CURRENT_HOSTNAME;

// Define a function to fetch and process sitemap
const fetchSitemap = async () => {
  try {
    // Get the parent directory of the script file
    const scriptParentDir = path.dirname(__dirname);

    // Fetch the sitemap.xml and robots.txt from a given HOSTNAME defined in the .env file
    const { data } = await axios.get(`${process.env.HOSTNAME}sitemap.xml`);

    // Parse the fetched sitemap.xml content
    xml2js.parseString(data, async (err, result) => {
      if (err) {
        console.error(err);
        return;
      }
      const sitemaps_arr = [];

      // Get the list of sitemaps from the parsed result
      const sitemapList = result.sitemapindex.sitemap;

      // Iterate through sitemap items and modify URLs
      for (const item of sitemapList) {
        if (item.loc[0].includes(targetURL)) {
          const { data } = await axios.get(item.loc[0]);
          let name = "";
          let modifiedLoc = "";

          if (item.loc[0].includes(targetURL)) {
            name = item.loc[0].split(targetURL)[1];
            modifiedLoc = item.loc[0].replace(targetURL, current_hostname);
          }
        }
      }
    });
  } catch (error) {
    console.error(error);
  }
};

With all the references updated, I proceeded to write our files to our public folder for Google to crawl.

The Launch

In the first week of November 2023, we confidently pushed our project live, transitioning smoothly into operation without significant disruptions.

However, "without major issues" came with a caveat. Many weeks after, the arrival of our Core Web Vitals (CWV) report revealed performance challenges that had lain dormant until the site was tested by real-world production demands and increased traffic. These insights prompted a focused effort to optimize our site’s performance post-launch and made us question How do we improve Page Load Times and Web Core Vitals in Nuxt

The aftermath

3 months after launch, our work on the redesign is helping drive a 40% increase in visits and a 20% increase in session duration to the already impressive numbers.

I hope this case study sheds light on the intricacies of such a migration and serves as a valuable resource for those considering a similar transition. The journey has been full of learning opportunities.

Thank you for sticking with me to the very end.

How to Improve Page Load Times and Web Core Vitals in Nuxt Websites

Christonit — Fri, 16 Feb 2024 18:28:32 +0000

We embarked in a website redesign using Nuxt that spanned 2 months of hardwork before launch. After a couple of weeks with our new website live, it was time to run an user experience and core web vitals audit. When the reports came in, we had 2 tasks.

How do we improve our web performance score to comply with Lighthouse, specifically, FCP and Total Blocking time when navigating between posts and how do we decrease the page load time when navigatig between posts.

After digging deep on our CWV reports and our codebase. I nailed down the main offenders of our bad performance scores and long load times when navigating between pages:

GraphQL queries for our flagship pages: Depending on the information we needed to fetch, GraphQL would take longer to respond.
Too many requests
Some iframes were not being correctly deferred

With these findings, I had 3 tactics that I could implement:

- Accelerating Resource Loading with Preconnection
- Asynchronous creation and loading of Embeds
- Migrating the main post request from GraphQL to REST

I have to clarify, before going further, that already had best practices such as image optimization or lazy loading are already in place but these I'm explaining below are just tactics we weren't initially implementing.

1. Accelerating Resource Loading with Preconnect

We use resources from many external domains such as google fonts and our content backend. When visiting a page as a new user, this often requires establishing connections to domains that are not initially connected to our main page's loading process. This connection time might mean a longer response time from the server.

We tackled this problem by establishing network connections early to our font servers and our backend in our layout page.



<template>
  <Head>
    <link rel="preload" as="font" />
    <link rel="preconnect" href="https://fonts.googleapis.com" />
    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
    <link rel="preconnect" :href="hostname" crossorigin />
  </Head>
</template>

This might mean less wait/queue time by the time we need to download those resources.

Pre-connection for font downloading might not be the best example, but pre-connecting avoids getting a performance penalty from google because font loading is something that could be render blocking or could induce cumulative layout shift.

2. Asynchronous component creation and lazy loading of Embeds

In some cases we have embeds that are most of the time above the fold, this means they execute too close to our initial Document load (DOMContentLoaded event), and we cannot defer them organically; they will initialize loading regardless of our intent to lazy load.

This has a negative side effect on our performance in the way that when our embed begins to load, it's going to initialize downloading all its dependencies.

In our case, on some pages, we have an embedded real-time stocks scanner that is its separate SPA (Single Page App) and most of the posts load a text-to-speed audio player.

The result is that so close to our initial load, our page is expediting the request for many resources almost equivalent to the download of a new page that is not critical for our FCP. This meant a variable amount of time from 100ms to more than 1 second depending on how the other server/domain is loaded at the moment of our request. There also could be the case where all this data is downloaded and executed alongside other heavy processes on our website, which could slow down a user's computer.

Google might penalize you by saying "You need to reduce your JS execution time or your initial payload is too big".

To workaround this issue I took 2 approaches:

Timing Lazy Loading the Iframes

Move the iframe to its out component and make its rendering conditional on the parent component.



<template>
  <!-- Conditionally render the iframe component based on the 'showIframe' variable -->
  <LazyTopGainerIframe v-if="showIframe" />
</template>

<script setup>

  const showIframe = ref(false);

  onMounted( async () => {
    // Check if we are on the client-side before setting the timeout
    if (process.client) {
      // Use setTimeout to delay setting 'showIframe' to true
      setTimeout(() => {
        showIframe.value = true;
      }, 3000); // 3-second delay
    }
  })

</script>

Dynamically Create the Iframe

Outside of our mounting lifecycle, we create a function that creates the iframe and assign its src attribute or have the iframe in place but instead of loading the src straight up, we create a data-src attribute and in our function we query for the iframe and switch data-src to src.



export const addScript = () => {
  let embedScript = document.createElement("script");
  embedScript.setAttribute("src", "https://platform.example.com/widgets.js");
  embedScript.setAttribute("defer", true);

  const body = document.querySelector("body");
  body.appendChild(embedScript);
};



onMounted(async () => {
  if (post.tags) {
    await useSidebarSorter(post.tags);
  }

  if (process.client) {
    await fetchLeftoverInformation();

    await setTimeout(() => {
      addScript();
    }, 1000);
  }
});

The most important part for this to work, is to make this function async and wrap our iframe generator or DOM query into a timeout we are awaiting. What this trick does is that we can add enough time to wait for execution that we know for sure that the DOM and other more important resources for our FCP (First Contentful Paint) or LCP (Large Contentful Paint) like images or fonts have finished loading. Making instructions a promise (async/await) makes our code concurrent, meaning waiting for the result of a certain function’s execution does not stop everything else from running.

With this approach, I’d suggest the biggest offenders (the elements that might represent the biggest performance budget) be executed as late as possible.

3. Migrating GraphQL to REST for our Nuxt - Headless Wordpress site

I noticed that depending on how big the query, the server response time varies. I played with the query a little bit and stripped it down multiple times to identify how much time everything takes to fetch.

But sadly the crucial information for our page load: post data such as title, id, SEO and Saswap Schema taxed too much the website.

That’s when I decided that the best course of action was to switch to rest for data fetching. Rest most of the time is 2x or 3x faster than GraphQL.

GraphQL allows us to query once for all the info we need but is slow, in this case a REST call brings most of the info we need to display the content above the fold. The only thing we need more calls for are the categories and comments.

I got stressed a little bit in the beginning because REST doesn’t let us fetch a post by slug and the comments don’t come in order.

The solution I came up with was a script to fetch all the posts and then made a dictionary with the slug as its key with the POST ID as its property.

Then I proceeded to create a small plugin to expose that information to our [..slug] page.

That way when we SPA Navigate, on our setup process I take the slug and search by that kay the POST ID on our Dictionary to do the REST call.

Post comments, category information, tags and related posts are fetched with GraphQL in our mounted hook on client side because they are not crucial for our initial page load. Our Mounted hook is Async so the execution and process of a function don’t block other processes.



const fetchLeftoverInformation = async () => {
    try {
        const { data } = await $fetch($config.public.hostname + "graphql", {
            method: "post",
            body: {
                query: `
            query NewQuery {
              postBy(slug: "${route.path}") {
                ${POST_CATEGORIES_QUERY}
                ${TAGS_QUERY}
                ${COMMENTS_QUERY}
              }
            }`
            }
        });

        if (data && data.postBy) {
            category.value = data.postBy.categories.nodes[0];
            comments.value = data.postBy.comments.nodes;
            tags.value = data.postBy.tags.nodes;
            commentsCount.value = data.postBy.commentsCount;
        }
    } catch (e) {
        // eslint-disable-next-line no-console
        console.error(e);
    }
};



onMounted(async () => {

    if (process.client) {
        await fetchLeftoverInformation();

    }

});

You might wonder Why a dictionary and not saving the post information in a JSON and filtering by id? Because checking by index is always going to yield the same time versus looping to find a specific item in the array where the time it takes to find an element grows linearly with the size of the array.

Conclusions

The average document load time (DOMContentLoaded evnet) and the full load for one of our most visited posts used to be 120 seconds and 150 seconds for full load.

Before

After

After this rework, the initial load is averaging 66 seconds.

These tactics helped achieve a wooping 45% decrease in our initial load time.

And I hope dearly that if you stumbled upon this article and you are facing similar issues on your projects, these tactics help you reach positive results as they have for me.