DEV Community: Oyedele Temitope

Why “Ship First” Fails in AI and How High-Performing Teams Build Instead

Oyedele Temitope — Tue, 20 Jan 2026 14:50:33 +0000

Most software teams grew up in a world where systems behave in black and white. A feature either works or it breaks, and when something fails, you trace the exact line of code and fix it. This mindset shaped how developers build, test and deploy applications.

AI systems do not behave this way. They answer with probabilities, not certainties. A response can be mostly correct, partly wrong or completely unpredictable. Quality shifts with context, data and even slight prompt changes. If teams carry the old “ship first, refine later” habit into an AI workflow, they end up with features that look promising in isolation but collapse under real user behaviour.

High-performing teams avoid this pattern by defining expected behaviour early, writing the tests that capture it and only then building the pipelines behind it. Their confidence comes from structure, not guesswork. The discipline looks simple from the outside: test first, deploy later. In practice, it is the clearest path to building AI systems that meet real user expectations.

In this article, we will take a look at how high-performing teams avoid that outcome. We will examine the discipline behind test-first development in the AI era, why it matters and how teams use it to build systems that behave consistently.

Why Traditional Engineering Logic Breaks in AI Systems

Software engineers often rely on a simple rule: if something breaks, there is a clear reason behind it. Code behaves the same way every time, so debugging becomes a matter of tracing the exact condition that failed. This creates a sense of certainty. Once you fix the problem, it stays fixed unless someone changes the code again.

AI systems do not offer that level of predictability. The same prompt can produce different answers at different times. A response may be mostly correct in one context and completely off in another. Two users can ask similar questions and receive different outcomes. The behaviour depends on data, phrasing, input structure and subtle changes that are impossible to map one by one.

This is why the “it works on my machine” mindset collapses in AI development. You cannot rely on a single example or a quick manual test to confirm that a feature is ready. Correctness must be judged across a range of inputs, not a single scenario. The goal is no longer to make the system perfect. The goal is to understand how it behaves across variations and whether that behaviour meets the standard your users expect.

High-performing teams recognise this early. They build tests that capture the spectrum of questions, contexts and edge cases the system must support. They do not search for one prompt that works. They search for consistent behaviour across many prompts, inputs and situations. This shift is the foundation for everything that follows, because you cannot design a reliable AI workflow without first accepting that its behaviour will always exist on a gradient.

You Don’t Need More Prompts. You Need a System

A common reaction when an AI feature behaves unpredictably is to write another prompt. If the output looks wrong, adjust a phrase. If the tone feels off, add a new instruction. If the model misunderstands a question, expand the context. This cycle repeats until the team finds a version that “seems to work.”

The problem is that this success rarely holds when real users arrive. A prompt that works during development may fail once the range of inputs expands. The team ends up patching and tweaking the system long after deployment because they were never testing for consistency in the first place.

High-performing teams avoid this trap. They do not chase perfect prompts. They build a system that helps them discover what actually works across a wide set of scenarios. Instead of guessing, they evaluate prompts against structured tests that represent true user behaviour. They measure how often a specific approach holds up, where it breaks and what patterns lead to reliable responses.

This is the moment where discipline replaces intuition. A prompt is no longer “good” because it produced a single correct answer. It is good because it performs consistently when tested at scale. This shift unlocks predictability, and it gives teams a clear path forward without relying on trial and error.

Start With the User, Not the Model

A lot of AI projects begin with the question, “Which model should we use?” It feels like the natural place to start, especially with the rapid pace of new releases. But model choice is rarely the real issue. A system fails because it does not match what users expect, not because the model was one version behind.

High-performing teams flip this process around. They begin by asking what users actually need from the system. If the feature is a support assistant, what kinds of questions will it handle? If it is a search tool, what signals matter most when ranking results? If it generates content, what tone, length or structure should the output follow? These expectations become the foundation for testing long before any model or prompt enters the picture.

This approach clarifies the goal. Instead of trying to guess what might work, the team defines the behaviours the system must demonstrate. Each behaviour becomes a test case. The tests then guide prompt design, retrieval logic, and eventual model selection. The model becomes a part of the solution, not the starting point.

That alignment prevents a common mistake: building something that looks impressive in isolation but fails when actual users interact with it. When tests come from user expectations, the system grows around real needs, not assumptions. It also becomes easier to measure progress, because every improvement shows up as a test that now passes.

Defining the Test Space for AI Systems

Before teams can test an AI system effectively, they need to agree on what “good behaviour” actually means in practice. That requires understanding the kinds of questions the system will receive, the scenarios that matter most, and the risks that must be controlled. High-performing teams define this test space early because it shapes every decision that follows.

Below are the key dimensions they use to define that space.

Coverage Metrics

Coverage metrics describe the distribution of real user queries. They show which categories appear most often, which ones carry higher value, and how different user segments behave. This helps teams decide where deeper testing is required and where lighter coverage is acceptable

Failure Mode Categories

AI systems fail in recognizable ways. Some responses hallucinate, others drift off-topic, and some ignore constraints or formatting rules. For example, a support assistant may confidently offer legal or medical advice when it should refuse, or a search tool may expose sensitive information if guardrails are not enforced. Identifying these failure modes early allows teams to design tests that surface problems before they reach users.

Guardrail Rules

Certain behaviours must remain stable regardless of context. These include safety boundaries, restricted topics, tone expectations, and output structure. Turning these requirements into tests helps prevent regressions as prompts, models, or data sources change.

Business-Critical Paths

Not all queries carry the same risk. Interactions tied to revenue, compliance, or customer trust require stronger guarantees. High-performing teams invest more testing effort in these paths to ensure they remain reliable as the system evolves.

Evaluation Metrics

Teams also decide how results will be measured. Relevance, accuracy, consistency, and latency are common criteria. In practice, these metrics are measured using a mix of automated techniques such as model-graded evaluations, deterministic checks like schema or constraint validation, and semantic similarity scoring rather than strict pass or fail assertions. Together, these metrics define what acceptable performance looks like for the system.

Building and Validating the System After Writing Tests

Once the test suite is clear, teams begin shaping the system around it. This is where prompts, retrieval steps and business logic come together to form the full workflow. The goal is simple: build the system, then check how well it holds up against the expectations defined in the tests.

Below are the steps high-performing teams focus on during this stage.

Pipeline Construction

The first step is assembling the core components of the AI workflow. This includes retrieval logic, classification steps, prompt templates, routing rules and any business-specific operations the feature requires. Each part supports a behaviour captured in the test suite, so the pipeline grows in a direction that reflects real user needs.

Real Data Validation

With the pipeline in place, teams run the system against real or representative data. This is where weaknesses appear. A response might be correct but too slow. A prompt may break when phrasing changes slightly. Formatting issues, missing context and inconsistent reasoning show up long before deployment when validation uses the right test cases.

Closing the Gaps the Tests Expose

Some issues only appear when the full workflow is tested across a broad set of scenarios. The system may drift from user intent, misunderstand certain categories or place the wrong weight on low-value tasks. These gaps guide the next round of adjustments. By refining the system at this stage, teams prevent misaligned behaviours from reaching production.

Best Practices Teams Use to Keep Their Systems Dependable

As teams refine their workflows, certain habits help maintain consistent behaviour and reduce surprises. These practices keep the system aligned with user expectations and provide guardrails as the system evolves. Below are the practices teams use to keep their systems dependable as they grow.

Keep humans in the loop to catch subtle issues that automated tests may miss and provide early feedback that improves the system’s alignment.
Turn failures into new test cases so problems become clear specifications and the system grows stronger with each improvement. Automate evaluations in your CI pipeline to catch regressions early and ensure every update meets the expected standard before it ships.
Version prompts, templates, datasets and routing logic so changes remain traceable and accidental regressions are easier to identify.
Re-test with real user samples to keep the system aligned with shifting behaviour patterns and uncover new edge cases.

Wrapping Up

AI systems behave on a spectrum, not in absolutes, which makes traditional “ship first, fix later” habits unreliable. The teams that build dependable AI features take a different path. They define how the system should behave, outline the space they need to test and shape their pipelines around those expectations. Testing comes first because it is the only way to bring predictability into a workflow built on probabilistic outputs.

While this approach can feel slower and more expensive than shipping quickly and fixing issues later, teams consistently find that structured evaluations reduce rework, limit regressions, and save engineering time over the life of the system.

This discipline does not slow teams down. It gives them clarity, reduces rework and prevents the common failure pattern where an AI feature looks promising in development but breaks under real user behaviour. By grounding their workflow in structured tests and refining the system against those tests, teams create AI features that stay aligned with user needs as they evolve.

Test first, deploy later. In the AI era, it is not just a process. It is how reliable systems are built.

15 Go Packages Worth Your Time

Oyedele Temitope — Thu, 26 Jun 2025 12:09:50 +0000

I’ve worked with Go for years, building web apps, CLIs, microservices, daemons, and all the in-between stuff nobody talks about. If you’ve worked with Go long enough, you know the standard library is solid, but you also know it’s not always enough.

Most problems you’ll run into have already been solved—cleanly, efficiently, and without the nonsense—by libraries maintained by people who know what they’re doing.

So instead of throwing out a list of “cool” Go packages, here’s a hand-picked set of 15 libraries that are actually worth using, based on real-world experience. These are things I’ve reached for again and again, and they’ve never let me down.

Core Utilities

If you’re building anything that touches config, flags, or maps—or you just want cleaner, testable code,these are the packages that’ll help you get it done.

1. `spf13/cobra`

The Cobra package is designed for building modern CLI applications. It supports subcommands, nested flag sets, command aliases, and generates help text automatically. It’s especially useful for CLI tools with multiple commands and a deeper structure.

Cobra integrates well with spf13/viper for managing configuration and environment variables, which makes it ideal for full-featured command-line applications.

2. `urfave/cli`

This is a lighter-weight alternative to Cobra. It focuses on the fast development of command-line apps without requiring much setup. It supports flags, nested commands, and custom help messages, but does so with a smaller surface area.

urfave/cli is well-suited for internal tools or smaller utilities where you want fast iteration without a large framework.

3. `stretchr/testify`

testify is a testing toolkit that adds better assertions, mock capabilities, and test suite support on top of Go’s built-in testing package. It’s commonly used for simplifying test logic and improving test readability.

It doesn’t replace Go’s testing.T, but works alongside it, giving you utilities like assert.Equal, require.NoError, and mock.Call.

4. `mitchellh/mapstructure`

The mitchellh/mapstructure package helps decode map[string]interface{} values, such as those returned by json.Unmarshal or config files, into typed Go structs. It supports custom tags, weakly typed inputs, and embedded structs.

It’s especially helpful when working with dynamic inputs, such as user-defined configurations or external API data. You’ll often see it used in tools like Terraform or Consul.

Unlike encoding/json, this focuses purely on map decoding, not marshaling/unmarshaling structured JSON.

5. `olekukonko/tablewriter`

tablewriter helps you render formatted tables in the terminal. It takes slices or arrays and prints them as aligned, readable tables, with support for borders, color, and alignment.

This is a great fit for CLI tools that require displaying lists, metrics, or tabular data. The package doesn’t deal with interactive UIs (like tview does), but it handles static output well.

Web & APIs

If you're building APIs, HTTP services, or JSON endpoints, these libraries improve routing, performance, and readability without locking you into rigid frameworks.

6. `gin-gonic/gin`

Gin is a high-performance HTTP web framework. It wraps Go’s standard net/http package and adds routing, middleware support, JSON validation, and error handling.

It’s known for its speed and simplicity. If you’re building RESTful APIs and want something faster and more ergonomic than raw net/http, Gin is a strong option.

7. `labstack/echo`

Echo is a full-featured web framework for Go. Like Gin, it offers routing and middleware, but it also includes built-in support for CORS, GZIP, JWT middleware, request validation, and more.

Echo is best when you want more “batteries included” functionality and are building something that benefits from built-in helpers (e.g., API versioning, structured error handlers).

While Gin and Echo overlap in purpose, Echo includes more out-of-the-box capabilities for production APIs.

8. `go-chi/chi`

Chi is a lightweight router that focuses on composability and idiomatic Go. It’s built around net/http and designed to be modular and minimal.

Chi is ideal if you want full control over your stack without adopting a full framework. It integrates well with standard middleware patterns and is often used in microservice setups.

Unlike Gin and Echo, Chi avoids introducing new abstractions around handlers or context. If you like the standard library but want better routing, Chi is a great pick.

9. `tidwall/gjson`

GJSON allows you to extract values from JSON strings using a simple path syntax like user.name, items.#.id, etc., without unmarshalling into structs.

It’s incredibly fast and useful in cases where you’re inspecting or filtering raw JSON logs, responses, or configs. This is a read-only tool; if you want to modify JSON, check out sjson from the same author.

Dev & Ops Helpers

These packages aren't about features—they're about keeping your codebase sane, your logs readable, and your CI clean. They’re the ones that make maintenance easier over time.

10. `golangci/golangci-lint`

The golangci-lint tool aggregates dozens of popular Go linters (like govet, staticcheck, errcheck, etc.) into a single, fast binary. It’s configurable, fast, and widely used in continuous integration (CI) pipelines.

Instead of manually juggling multiple linters, golangci-lint provides a unified interface to catch real issues early.

11. `sirupsen/logrus`

Logrus is a structured logger for Go that outputs in JSON, text, or custom formats. It’s simple to use and integrates easily with most projects.

While it’s not the fastest logger around, it’s stable, well-documented, and good for most apps that aren’t logging at extreme scale.

12. `uber-go/zap`

Zap is a structured logger built for speed. It avoids reflection and allocates far less than most logging packages.

If you’re working on high-throughput systems, background workers, or want better performance under load, Zap is a better choice than Logrus. Its API is more verbose, but the tradeoff is worth it for larger systems.

Storage, Persistence, and Data

If you’re storing state, caching data, or working with local files, these packages help avoid unnecessary complexity or external dependencies.

13. `jmoiron/sqlx`

sqlx builds on Go’s database/sql and adds useful features like named queries, struct mapping, and scanning slices.

It doesn’t try to be an ORM—it just smooths out some of the rough edges. Perfect for apps that interact directly with SQL and want less boilerplate.

14. `etcd-io/bbolt`

Bolt, now maintained by etcd-io, is a low-level, embedded key-value store. It’s ACID-compliant, fast, and doesn’t require a server process.

It’s a solid choice for storing user preferences, cached data, or small records locally. If you need a lightweight persistence layer without a full DB engine, Bolt is a great fit.

15. `go-yaml/yaml`

The go-yaml/yaml is the most reliable YAML parser for Go. It supports the full YAML spec (including anchors and aliases) and handles real-world files used in infra tools.

You’ll often run into YAML if you work with Kubernetes, Docker Compose, or Terraform. This package helps you load and parse them properly without surprises.

Concusion

These 15 packages are ones I’ve used repeatedly across real projects, and I keep coming back to them for a reason. They’re maintained, documented, and solve problems without getting in your way.

If you’ve got a favorite package I didn’t list—especially niche tools—drop a comment. Always happy to find sharp tools.

How to Scrape Products from a Page with Infinite Scroll via 'Load More' Button

Oyedele Temitope — Mon, 24 Mar 2025 18:01:33 +0000

Have you ever wanted to scrape data but didn’t know where to start? Or have you struggled with websites that hide extra data behind interactions like a "Load More" button? Don’t worry; web scraping doesn’t have to be complicated, and you don’t need to be a pro to get started.

In this tutorial, you’ll learn how to scrape data in Node.js. We’ll use tools like Puppeteer and Cheerio to extract data from the website.

Prerequisites

To follow along with this tutorial, you’ll need to meet the following prerequisites:

Basic knowledge of JavaScript and Node.js
Node.js installed
A code editor
Basic knowledge of web scraping

Project Scope

For this project, we’ll scrape products along with the necessary information about the products from this demo website. You can check it out to get familiar with

Below are some of the libraries needed for scraping the data in this project:

Puppeteer: For the project, it will be used to scrape data and simulate browser interactions, such as the “Load More” button.
Cheerio: For this project, it will be used to parse and extract data.
Csv-writer: exporting data into a structured CSV file
Set Up the Project

Open your terminal and navigate to the folder where you want this project set up. Initialize a new Node.js project by running this command:

npm init -y

Next, install the required libraries:

npm install puppeteer cheerio csv-writer

Access the Content

First, check out the site from which you want to extract the data for this project. Below is an image of what the webpage you’ll extract data from looks like:

You’ll notice that 12 products are displayed on the webpage. We're focusing on extracting these 12 products. This will be achieved using CSS selectors.

Go back to the demo webpage. Right-click on an element and select Inspect to open the Developer Tools. Alternatively, you can open the Developer Tools by pressing Ctrl+Shift+I (or Cmd+Option+I on Mac).

Locate the container for all products. In this case, it's a div with the ID #product-grid. Copy the selector.

The selector you just copied will be used to access the content.

In order to extract the product grid's HTML structure, Puppeteer will be used to navigate to the demo webpage and extract the product grid’s HTML using the #product-grid selector.

In your project folder, create a file called scrape.js. Inside it, add the following code:

const puppeteer = require("puppeteer");

(async () => {
  const url = "https://www.scrapingcourse.com/button-click"; 
  const browser = await puppeteer.launch({ headless: true }); 
  const page = await browser.newPage();

  await page.goto(url, { waitUntil: "domcontentloaded" });

  await page.waitForSelector("#product-grid"); 
  const productGridHTML = await page.evaluate(() => {
    return document.querySelector("#product-grid").outerHTML;
  });

  console.log("Initial product grid HTML:", productGridHTML); 

  await browser.close();
})();

The await page.waitForSelector ensures the #product-grid element is loaded. This guarantees the content is available before scraping. It then uses page.evaluate to extract the HTML structure of the #product-grid and logs the result to the console.

Run your code using this command:

node scrape.js

You’ll see an output in your terminal printing the HTML structure of the #product-grid, which contains the first 12 products:

Load More Products

Each time you click the “Load More” button, it displays more products. This means there are still other products that have yet to be scraped. To achieve this, you need to simulate clicking the button multiple times to retrieve all the available products.

You first need to identify its selector, so go back to the demo webpage. Right-click on the "Load More" button and select Inspect. The HTML code for the button will then be highlighted:

In the Developer Tools, you’ll see the button's structure:

<button id="load-more-btn" data-offset="10">Load More</button>

This tells you two important things: First, you can uniquely identify the button using the id attribute, which is #load-more-btn.

Secondly, the data-offset attribute keeps track of the number of products loaded. After each click, the data-offset value changes to reflect the updated count.

Select Copy > Copy selector to use the selector in your code.

To ensure new products are fully loaded before scraping, we’ll use await page.waitForFunction. This waits for the data-offset value of the "Load More" button to update.

Update the logic in the async function as follows:

(async () => {
  const url = "https://www.scrapingcourse.com/button-click";
  const browser = await puppeteer.launch({ headless: true }); 

  const page = await browser.newPage();
  await page.goto(url, { waitUntil: "domcontentloaded" });
  await page.waitForSelector("#load-more-btn");
  for (let i = 0; i < 5; i++) {
    const currentOffset = await page.evaluate(() => {
      return document
        .querySelector("#load-more-btn")
        .getAttribute("data-offset");
    });
    await page.click("#load-more-btn"); 
    await page.waitForFunction(
      (prevOffset) => {
        const button = document.querySelector("#load-more-btn");
        return button && button.getAttribute("data-offset") !== prevOffset;
      },
      { timeout: 5000 },
      currentOffset,
    );
  }
  const productGridHTML = await page.evaluate(() => {
    return document.querySelector("#product-grid").outerHTML;
  });
  console.log("Updated product grid HTML:", productGridHTML); 
  await browser.close(); 
})();

The main logic introduced here is the ability to handle product dynamic loading. Each time the button is clicked, the data-offset attribute of the button is tracked to verify that new content has been loaded.

You'll also notice a for loop in the logic. it used to click the button multiple times. after each click the await page.waitForSelector waits for the data-offset value to update, until it disappears which indicates that there are no more products availble to be displayed.

For additional techniques on handling dynamic content, check out how to do web scraping with Puppeteer and NodeJS.

Your code should now look like this:

const puppeteer = require("puppeteer");

(async () => {
   const url = "https://www.scrapingcourse.com/button-click"; 
   const browser = await puppeteer.launch({ headless: true }); 
    const page = await browser.newPage();
   await page.goto(url, { waitUntil: "domcontentloaded" });
   await page.waitForSelector("#load-more-btn"); 

   for (let i = 0; i < 5; i++) {
     const currentOffset = await page.evaluate(() => {

         .querySelector("#load-more-btn")
         .getAttribute("data-offset");
     });
     await page.click("#load-more-btn"); 
     await page.waitForFunction(
       (prevOffset) => {
         const button = document.querySelector("#load-more-btn");
         return button && button.getAttribute("data-offset") !== prevOffset;
       },
       { timeout: 5000 }, 
       currentOffset,
     );
   }
   const productGridHTML = await page.evaluate(() => {
     return document.querySelector("#product-grid").outerHTML;
   });
   console.log("Updated product grid HTML:", productGridHTML); 
   await browser.close(); 
 })();

Re-run the code, and you’ll see all the products loaded and displayed in your terminal:

Parse Product Information

Now that the full HTML of the products has been captured, the next step is to parse the product details. This is where Cheerio comes in. It will be used to parse the HTML of the product grid and extract specific details for each product, such as name, price, and image URL, into a JSON file. The .product-item selector will be used to help locate individual products.

Import the Cheerio and fs library:

const cheerio = require("cheerio"); // Import Cheerio for parsing HTML
const fs = require("fs"); // Import the file system module for saving data

Update the logic in the async function to this:

(async () => {
  const url = "https://www.scrapingcourse.com/button-click";   const browser = await puppeteer.launch({ headless: true });

  const page = await browser.newPage();

  await page.goto(url, { waitUntil: "domcontentloaded" });
  await page.waitForSelector("#load-more-btn"); // Wait for the "Load More" button to be available

  for (let i = 0; i < 5; i++) {
    const currentOffset = await page.evaluate(() => {

      return document
        .querySelector("#load-more-btn")
        .getAttribute("data-offset");
    });

    await page.click("#load-more-btn");

    await page.waitForFunction(
      (prevOffset) => {
        const button = document.querySelector("#load-more-btn");
        return button && button.getAttribute("data-offset") !== prevOffset;
      },
      { timeout: 5000 }, 
      currentOffset,
    );
  }

  const productGridHTML = await page.evaluate(() => {
    return document.querySelector("#product-grid").outerHTML;
  });

  const $ = cheerio.load(productGridHTML); 
  const products = []; 


  $(".product-item").each((_, element) => {
    const name = $(element).find(".product-name").text().trim();
    const price = $(element).find(".product-price").text().trim();
    const image = $(element).find(".product-image").attr("src");
    const url = $(element).find("a").attr("href");

    const fullImageUrl = image.startsWith("http")
      ? image
      : `https://www.scrapingcourse.com${image}`;
    const fullUrl = url.startsWith("http")
      ? url
      : `https://www.scrapingcourse.com${url}`;

    products.push({
      name,
      price,
      image: fullImageUrl,
      url: fullUrl,
    });
  });

  // Save the product details to a JSON file
  fs.writeFileSync("products.json", JSON.stringify(products, null, 2));
  console.log("Product data saved to products.json");

  await browser.close(); 
})();

Since the full HTML of the product has already been scraped into productGridHTML all that Cheerio needs to do is process the productGridHTML by converting it into a DOM-like structure in memory. This enables queries and manipulations like how you would use CSS selectors in a browser. Cheerio uses .product-item to locate each product container and .product-name, .product-price, .product-image, and <a> elements to retrieve specific details.

Your code should now look like this:

const puppeteer = require("puppeteer");
const cheerio = require("cheerio"); 
const fs = require("fs"); 

(async () => {
   const url = "https://www.scrapingcourse.com/button-click"; 
   const browser = await puppeteer.launch({ headless: true }); 
    const page = await browser.newPage();
    await page.goto(url, { waitUntil: "domcontentloaded" });
   await page.waitForSelector("#load-more-btn"); // Wait for the "Load More" button to be available

   for (let i = 0; i < 5; i++) {
     const currentOffset = await page.evaluate(() => {

       return document
         .querySelector("#load-more-btn")
         .getAttribute("data-offset");
     });
      await page.click("#load-more-btn");
      await page.waitForFunction(
       (prevOffset) => {
         const button = document.querySelector("#load-more-btn");
         return button && button.getAttribute("data-offset") !== prevOffset;
       },
       { timeout: 5000 }, 
       currentOffset,
     );
   }

   const productGridHTML = await page.evaluate(() => {
     return document.querySelector("#product-grid").outerHTML;
   });
    const $ = cheerio.load(productGridHTML); 
   const products = []; 

   $(".product-item").each((_, element) => {
     const name = $(element).find(".product-name").text().trim();
     const price = $(element).find(".product-price").text().trim();
     const image = $(element).find(".product-image").attr("src");
     const url = $(element).find("a").attr("href");

     const fullImageUrl = image.startsWith("http")
       ? image
       : `https://www.scrapingcourse.com${image}`;
     const fullUrl = url.startsWith("http")
       ? url
       : `https://www.scrapingcourse.com${url}`;

     products.push({
       name,
       price,
       image: fullImageUrl,
       url: fullUrl,
     });
   });

   fs.writeFileSync("products.json", JSON.stringify(products, null, 2));
   console.log("Product data saved to products.json");
    await browser.close(); 
 })();

Re-run the code, and you’ll have a product.json displaying the details like this:

You’ve successfully scraped a dynamic website, Node.js, using Puppeteer and Cheerio. Now, you can decide to stop here or go further and convert the JSON file to CSV. If you decide to stop here, no problem, but if you decide to continue, then let’s get on with it!

Export Product Information to CSV

Import the csv-writer library:

const createCsvWriter = require("csv-writer").createObjectCsvWriter;

Import the csv-writer library:

Next, add this code inside the async function right before the await browser.close:

const csvWriter = createCsvWriter({
   path: "products.csv", 
   header: [
     { id: "name", title: "Name" },
     { id: "price", title: "Price" },
     { id: "image", title: "Image URL" },
     { id: "url", title: "Product URL" },
   ],
 });

 const jsonData = JSON.parse(fs.readFileSync("products.json", "utf-8"));
 await csvWriter.writeRecords(jsonData);
 console.log("Product data successfully written to products.csv");

Now your code looks like this:

const puppeteer = require("puppeteer"); 
const cheerio = require("cheerio"); 
const fs = require("fs"); 
const createCsvWriter = require("csv-writer").createObjectCsvWriter; 

(async () => {
 const url = "https://www.scrapingcourse.com/button-click"; 
 const browser = await puppeteer.launch({ headless: true }); 

 const page = await browser.newPage();

 await page.goto(url, { waitUntil: "domcontentloaded" });
 await page.waitForSelector("#load-more-btn"); 

 for (let i = 0; i < 5; i++) {
   const currentOffset = await page.evaluate(() => {
       .querySelector("#load-more-btn")
       .getAttribute("data-offset");
   });

   await page.click("#load-more-btn");

   await page.waitForFunction(
     (prevOffset) => {
       const button = document.querySelector("#load-more-btn");
       return button && button.getAttribute("data-offset") !== prevOffset;
     },
     { timeout: 5000 }, 
     currentOffset
   );
 }

 const productGridHTML = await page.evaluate(() => {
   return document.querySelector("#product-grid").outerHTML;
 });

 const $ = cheerio.load(productGridHTML); // Load the HTML into Cheerio for parsing
 const products = []; 

 $(".product-item").each((_, element) => {
   const name = $(element).find(".product-name").text().trim();
   const price = $(element).find(".product-price").text().trim();
   const image = $(element).find(".product-image").attr("src");
   const url = $(element).find("a").attr("href");

   const fullImageUrl = image.startsWith("http")
     ? image
     : `https://www.scrapingcourse.com${image}`;
   const fullUrl = url.startsWith("http")
     ? url
     : `https://www.scrapingcourse.com${url}`;
   products.push({
     name,
     price,
     image: fullImageUrl,
     url: fullUrl,
   });
 });

 fs.writeFileSync("products.json", JSON.stringify(products, null, 2));
 console.log("Product data saved to products.json");
 const csvWriter = createCsvWriter({
   path: "products.csv", 
   header: [
     { id: "name", title: "Name" },
     { id: "price", title: "Price" },
     { id: "image", title: "Image URL" },
     { id: "url", title: "Product URL" },
   ],
 });

 const jsonData = JSON.parse(fs.readFileSync("products.json", "utf-8"));
 await csvWriter.writeRecords(jsonData);
 console.log("Product data successfully written to products.csv");

 await browser.close();
})();

Re-run your code, and you’ll see that a products.csv has been created. It contains all the data that was scraped.

Now we’re finally done!

That’s How To Scrape Data in Node.js using Puppeteer and Cheerio

In this tutorial, we explored how to scrape data in Node.js using Puppeteer and Cheerio. With the right tools, web scraping can be effective and approachable, even for beginners.

Remember to always respect the ethical boundaries of web scraping and adhere to the terms of service of the websites you work with. To ensure you’re scraping responsibly, check out Web Scraping Best Practices and Tools 2024 for more information.

How to create an AI Avatar in Go using Simli

Oyedele Temitope — Thu, 16 Jan 2025 16:56:18 +0000

Creating a real-time video AI avatar for interaction is a great way to blend technology and human-like communication. The exciting part is that it can be used in various aspects, such as customer support, virtual assistants, education, and more.

Simli provides this. It is an AI video avatar-generating platform that allows you to create lip-synced AI avatars with lifelike, animated characters, realistic head movements, and synchronized speech. Now, you can utilize its product and build in Go!

In this tutorial, we’ll take a look at how to generate an AI avatar in Go using Simli.

Prerequisites

Before we proceed, ensure that you have the following:

Basic knowledge of Go and its concepts.
Go version 1.22 or higher.
A Simli account.
An audio file.

Set up your Environment

To get started, go to your dashboard and navigate to the API KEYS section. There, you’ll see your API key. Copy it and store it somewhere, as you’ll need it during the building process.

Next, you’ll need an AI avatar. You can choose to create your own avatar or choose from the list in your dashboard. For the project, we’ll choose an avatar from the one assigned to us. Navigate to your avatars. There, you’ll see the list of avatars you have:

Copy the ID of your desired avatar and store it somewhere.

Now you’re ready to start building!

Build the Project

Open up a terminal in your code editor and navigate to the folder for the project.

You need to initialize a new Go module. Run this command:

go mod init simli-avatar

Next, you need to install Simli sdk. While there isn't an official Simli sdk for Go yet, There’s a Go library you can use. Add this library to your project using this command:

go get github.com/oyedeletemitope/simli-go-sdk

This will install the library that you’ll need to use Simli.

Import Required Packages and Set Data Structure for Response

You need to import the necessary packages for file handling, HTTP communication, JSON encoding/decoding, environment variables, and working with Simli's SDK. Create a main.go file and add the following code:


import (
   "bytes"
   "encoding/base64"
   "encoding/json"
   "fmt"
   "log"
   "net/http"
   "os"
   "os/exec"
 "github.com/oyedeletemitope/simli-go-sdk/client"
 "github.com/oyedeletemitope/simli-go-sdk/models"
)

Below is the list of packages imported and what they do:

encoding/base64: To encode binary data into Base64 format.
encoding/json: To work with JSON data.
fmt: For formatted I/O operations.
log: For logging errors or information.
net/http: To handle HTTP server and client operations.
os: For file and environment operations.
simli-go-sdk: The library for interacting with Simli's API.

You also need to create the data structure for the response:


type GenerateAvatarResponse struct {
        HLSURL string `json:"hls_url"`
        MP4URL string `json:"mp4_url"`
}

Here, we created a struct that defines the structure of the HTTP response sent back to the client.

Generate the AI Avatar

You need to create a handler to generate the avatar. It will handle HTTP requests for avatar generation by processing a PCM file and sending it to the Simli API. Add this code:


func generateAvatarHandler(w http.ResponseWriter, r *http.Request) {
        apiKey := "your-simli-api-key"
        faceID := "6ebf0aa7-6fed-443d-a4c6-fd1e3080b215"
        // Path to the PCM file
        pcmFile := "./audio/audio.pcm"
        // Read the PCM file
        audioData, err := os.ReadFile(pcmFile)
        if err != nil {
                http.Error(w, fmt.Sprintf("Failed to read PCM audio file: %v", err), http.StatusInternalServerError)
                return
        }
        // Convert PCM audio to Base64
        audioBase64 := base64.StdEncoding.EncodeToString(audioData)
        // Initialize Simli client
        c := client.NewClient(apiKey)
        // Prepare audio-to-video request
        audioReq := models.AudioToVideoRequest{
                SimliAPIKey:       apiKey,
                FaceID:            faceID,
                AudioBase64:       audioBase64,
                AudioFormat:       "pcm16",
                AudioSampleRate:   16000,
                AudioChannelCount: 1,
        }
        // Generate avatar
        resp, err := c.AudioToVideo(audioReq)
        if err != nil {
                http.Error(w, fmt.Sprintf("Failed to generate avatar: %v", err), http.StatusInternalServerError)
                return
        }
        // Respond with video URLs
        json.NewEncoder(w).Encode(GenerateAvatarResponse{
                HLSURL: resp.HLSURL,
                MP4URL: resp.MP4URL,
        })
}

Add the Main Function

Lastly, on the main.go file, you need to create the main function that’ll take in the avatar generation function, set up the HTTP server, define routes, and start serving API requests and static frontend files. Add the code right after the avatar generation function:


func main() {
   // Serve the generate-avatar endpoint
   http.HandleFunc("/generate-avatar", generateAvatarHandler)

   // Serve static HTML for the frontend
   fs := http.FileServer(http.Dir("./static"))
   http.Handle("/", fs)

   // Start the server
   port := ":8080"
   fmt.Printf("Server is running at http://localhost%s\n", port)
   log.Fatal(http.ListenAndServe(port, nil))
}

The code above initializes and starts an HTTP server that handles requests for the application. It defines the /generate-avatar route, which links to the handler responsible for processing audio files and generating avatar videos via the Simli API.

It also serves static HTML files from the ./static directory (which we’ll create shortly) to provide a simple frontend interface for users. Finally, the server runs on port 8080, which logs its address for easy accessibility to test or deploy.

Create the Static File

Now, you need to create the static file to serve as the frontend interface. Create a folder called static, and inside it, create an index.html file. Add this:


<!DOCTYPE html>
<html lang="en">
 <head>
   <meta charset="UTF-8" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
   <title>Avatar Video</title>
   <script src="https://cdn.tailwindcss.com"></script>
 </head>
 <body
   class="bg-gray-100 min-h-screen flex flex-col items-center justify-center p-4"
 >
   <div class="max-w-lg w-full bg-white shadow-lg rounded-lg p-6 space-y-6">
     <h1 class="text-2xl font-bold text-gray-800 text-center">
       Generated Avatar Video
     </h1>
     <button
       id="generate"
       class="w-full bg-blue-500 hover:bg-blue-600 text-white font-semibold py-2 px-4 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-400"
     >
       Generate Avatar
     </button>
     <div>
       <h2 class="text-lg font-semibold text-gray-700 text-center">
         Video Playback
       </h2>
       <video
         id="avatar-video"
         controls
         class="w-full mt-4 rounded-lg hidden"
       ></video>
     </div>
   </div>

   <script>
     document
       .getElementById("generate")
       .addEventListener("click", async () => {
         try {
           // Fetch the video URLs from the backend
           const response = await fetch("/generate-avatar", { method: "GET" });

           if (!response.ok) {
             alert("Failed to generate avatar");
             return;
           }

           const data = await response.json();

           // Display the video in the browser
           const videoElement = document.getElementById("avatar-video");
           videoElement.src = data.mp4_url; // Use MP4 URL for playback
           videoElement.classList.remove("hidden");
         } catch (error) {
           console.error("Error:", error);
           alert("An error occurred while generating the avatar");
         }
       });
   </script>
 </body>
</html>

Test the application

Now that we’re done, let’s test the application by starting it up. Run this command:

go run main. Go

Open http://localhost:8080/ in your local browser and test it out:

Wasn't so hard, was it?

That's how to create an AI avatar in Go using Simli

In this tutorial, we looked at how to create an AI avatar in Go through a library that uses Simli sdk for AI. It's important to note that the sdk is still in early development, and more Simli features are still being added.

Please share if you found this helpful.

Forward Proxy vs Reverse Proxy vs Load Balancers

Oyedele Temitope — Mon, 23 Dec 2024 16:20:18 +0000

Have you ever wondered how some of the world’s largest websites handle millions of users simultaneously without crashing or how they transfer your data securely when directing you to the right server?

That's because of some behind-the-scenes tools and components like forward proxies, reverse proxies, and load balancers. These components are considered part of the backbone of modern web infrastructure. It helps organizations scale, secure, and optimize their systems.

In this tutorial, we'll examine these three components and break down each of them in simple terms to show how they work and why they matter.

What Is a Forward Proxy?

A forward proxy is a server that acts as an intermediary between a client and the webserver holding the desired data.

Imagine you’re planning to dine at a popular restaurant but do not want to interact directly with the staff, probably because you're too busy, shy, or for reasons best known to you. Instead, you have a personal assistant who makes the reservation on your behalf. The restaurant staff never interacts directly with you; they only communicate with your assistant.

You can liken it to the way forward proxies work. In this case, you are acting like the laptop used to browse the internet, and your personal assistant is the forward proxy server. A forward proxy server acts as a middleman between your private network and the public internet, where your request goes out.

Why Forward Proxies Are So Important

Some of the reasons why forward proxies are so important include:

Protection and Filtering: A forward proxy can act as a means of protection by filtering traffic and also blocking any harmful website, scripts, or even code before the response gets forwarded back to you.
Logging and Monitoring: A forward proxy has the ability to log user activities. It can show which websites are being visited and by whom. This comes in handy if you're looking to identify any form of malicious activity.
Caching Responses: One of the cool thing about a forward proxy is that it caches responses. Let's say, for example, you want to access a resource from a website. Forward proxy will fetch the resource and cache it locally. If another user requests to access the same resource, the forward proxy delivers the cached copy without needing to fetch it from the web again. This way, it saves bandwidth and reduces any unnecessary traffic to the internet.

What Is a Reverse Proxy?

As the name suggests, a reverse proxy is a type of proxy server, but in reverse. Instead of managing outgoing requests from a client like the way a forward proxy does, a reverse proxy manages incoming requests on behalf of backend servers.

Let’s revisit the restaurant analogy. After your assistant makes the reservation and you arrive at the restaurant, you check in at the reception desk instead of searching for a free table by yourself. The receptionist takes you to the appropriate table to ensure that the restaurant's dining area is organized, efficient, and not overcrowded. In this analogy, the receptionist is acting as a reverse proxy.

A reverse proxy sits in front of the backend servers and handles incoming client requests. It then distributes these requests to the appropriate servers based on load, availability, or other criteria.

Why Use a Reverse Proxy?

Some of the reasons why you should use a reverse proxy include:

Load Balancing: You can use a reverse proxy to distribute incoming traffic across many backend servers to ensure that no single server is overworked. It also improves the performance and reliability of the application.
Improved Security: A reverse proxy conceals the backend servers' details from clients and acts as a single point of entry. It can also handle encryption, such as SSL/TLS, to ensure clients have a secure connection to the service.
Centralized Traffic Management: A reverse proxy gives you an overview of all incoming traffic; thus, you have better control, monitoring, and optimization.
Caching and Compression: Like forward proxies, reverse proxies can also cache responses coming from the backend servers to reduce the load and improve response times for frequently accessed resources.

What Is a Load Balancer?

A load balancer can be described as a tool or device that distributes incoming network or application traffic evenly across many servers. It ensures no single server is overwhelmed and helps maintain performance, availability, and reliability.

Still using the restaurant illustration, after the receptionist directs you to the dining area, you don’t approach a waiter randomly to place your order. Instead, there would be a floor manager who assigns you a waiter who is less busy to attend to you. This will allow every waiter to handle a fair number of guests and ensure that none of the waiters gets overworked, thus keeping the dining experience smooth.

In this scenario, you the customer, represent the incoming requests, the waiters are the backend servers handling those requests, while the floor manager is the load balancer, who makes sure that the distribution is even and the operation is smooth.

Why Use a Load Balancer?

You should use a load balancer for the following reasons:

Efficient Traffic Distribution: Just as the floor manager ensures that no single waiter is tasked with too much work, a load balancer does something similar by distributing traffic evenly across many servers, which ensures a smooth and efficient operation.
Failover and Redundancy: Load balancers help avoid situations where there is an interruption in service by redirecting traffic to an available server when a server goes down.
Improved Performance: A load balancer is used to improve performance as it balances traffic in order to maintain a consistent user experience even during peak hours.
Scalability: Load balancers support scaling by integrating new servers easily as demand increases, and this makes them ideal for applications.

Difference Between Forward Proxies, Backward Proxies, and Load Balancers

Forward proxies, reverse proxies, and load balancers all act as high-level intermediaries in network communication, but they serve different purposes and are positioned in different parts of the infrastructure.

When it comes to the direction of traffic, forward proxies manage outbound traffic from clients, reverse proxies, on the other hand, manage inbound traffic to servers, and load balancers distribute incoming traffic among multiple servers.

Also, forward proxies focus on client anonymity and filtering, reverse proxies optimize and secure server traffic, and load balancers prioritize traffic distribution and server efficiency.

Regarding security, forward proxies protect users, reverse proxies protect servers, and load balancers ensure reliability without a primary focus on security.

The table below shows detailed information about their differences:

Feature	Forward Proxy	Reverse Proxy	Load Balancer
Primary Purpose	Intermediary for client requests to external servers.	Intermediary for external client requests to backend servers.	Distributes traffic across multiple servers to balance load.
Client Interaction	Represents the client to the web server.	Represents the server to the client.	Does not directly represent either but focuses more on balancing.
Traffic Direction	Handles outgoing requests from clients to servers.	Handles incoming requests to servers.	Manages incoming requests and balances load evenly across servers.
Security Focus	Protects clients by hiding their identities while simultaneously filtering traffic.	Protects backend servers by hiding their details and also helps manage encryption.	Offers less in terms of security but can be integrated with firewalls and SSL termination.
Caching	Caches frequently accessed content locally to save bandwidth.	Caches server responses to improve performance.	Rarely caches but focuses on routing and distribution.

Conclusion

In this article, we examined three behind-the-scenes components of modern web infrastructure. We examined what they each do, why they are important, and their differences.

While each performs a different function, when all put together, they can optimize traffic flow, enhance security, and make web applications scalable and reliable. Knowing how they work and how to use them would help build efficient and high-performing systems.

How to Set Up Authorization in a Bookstore Management System with Go, HTMX, and Permit.io

Oyedele Temitope — Fri, 29 Nov 2024 09:41:57 +0000

Authorization is important when building applications, as it determines what actions and resources a user is allowed to access after they authenticate.

In this article, we'll take a look at how to implement authorization using permit.io. To demonstrate it, we'll be building a simple bookstore app using Golang and HTMX (I'm a huge fan).

Table of Content

Prerequisites
Project Scope
Project Setup
- Setup the Database
- Install Dependencies
- Setup your PDP (Policy Decision Point) Container
Build the Application
- Configure Database Connection
- Create the Handlers
- Create Authorization Middleware
- Create the Models
- Create the HTML Templates
Test the Application
Conclusion

Prerequisites

To follow through this tutorial, the following prerequisites should be met:

Golang installed along with a basic understanding of it.
A Permit.io account.
Docker installed.
Basic understanding of HTML, HTTP, and REST APIs.
PostgreSQL (database).
Familiarity with SQL.

Project Scope

For this demonstration, we'll keep things simple. We'll have two user types, an admin and a standard user. Both will be registered on Permit.io. Upon login, the database will consult Permit.io to determine the user's role and authorize their actions.

All users (admins included) can read books. Admins can also add, delete, and update books. Standard users are limited to reading books.

This tutorial will guide you through setting up a bookstore application with basic authorization. We’ll implement:

Authorization Logic: Define roles (Admin and Standard User) using Permit.io to restrict or grant access to different resources.
Database: Set up a PostgreSQL database to store book and user data.
Handlers: Implement routes for viewing, adding, updating, and deleting books with access control checks.
Frontend: Use HTMX to load book data dynamically.

Project Setup

In setting up the project, we’ll start by setting up permit.io. Navigate to your dashboard workspace and create a new project. I’ll give mine the name of bookstore.

This will create two environments: a development environment and a production environment.

Since we’re working locally, we’ll use the development environment. Click on Open dashboard in the Development environment and then on Create Policy. You’ll be asked to create a new resource first. Click create resource. Give it a name and state the actions. For this project, I’ll name mine books, and the action will be create, update, delete, and view.

Next, navigate to the policy editor section. By default, you should see an admin role already created. You just need to tick the view action we added, as it is not recognized by default. You need another role. This will be for users with only permission to read.

Click Create then Role and give it the name of user. Once created, you should see it in the policy editor and tick view in the user role you just created like so:

The next thing is to register the users who would be authorized by permit.io. Navigate back to your home menu through the sidebar menu you should still have something like this:

Click on Add users and then add, then add user. Fill in the details that will correspond to your users in the database.

Once that is done, navigate back to your project. In the Development For the environment for the bookstore project, click on the 3 dotted icon. You’ll see an option to copy the API key. Copy and save it somewhere, as you’ll be needing it for the project.

Setup the Database

Create a PostgreSQL database called bookstore. You’ll need to set up two tables:

users table: Stores user credentials and roles:


CREATE TABLE users (
  id SERIAL PRIMARY KEY,
  username VARCHAR(255) NOT NULL,
  password_hash VARCHAR(255) NOT NULL,
  role VARCHAR(50) NOT NULL
);

Go ahead and populate this, but make each user have a role of admin and user, respectively, and make sure they match the users added on Permit.io.

books table: Stores book details:


CREATE TABLE books (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  title VARCHAR(255) NOT NULL,
  author VARCHAR(255) NOT NULL,
  published_at DATE,
  created_at TIMESTAMPTZ DEFAULT now()
);

You don’t need to populate this we’ll be doing that in the code.

Install Dependencies

You’ll need to install the following dependencies:

github.com/permitio/permit-golang: Provides tools for handling role-based access control (RBAC) and permission management with Permit.io in Go applications.
github.com/google/uuid: This provides functions to generate and work with universally unique identifiers (UUIDs).
github.com/gorilla/mux: Helps to Implement an HTTP request router and dispatcher for handling routes in a web application.
github.com/joho/godotenv: This loads environment variables from a .env. file into the application, making it easier to manage configuration settings.
github.com/lib/pq: This is Go’s Postgres driver for communicating with PostgreSQL databases.
golang.org/x/crypto: Implements supplementary cryptographic algorithms and libraries that are not included in Go's standard library.

To install these dependencies, you need to initialize initializes a new Go module. This is the starting point for dependency management in Go.

Run this command:

go mod init bookstore

Next, run this command:


go get github.com/google/uuid \
       github.com/gorilla/mux \
       github.com/joho/godotenv \
       github.com/lib/pq \
       github.com/permitio/permit-golang \
       golang.org/x/crypto

This will install all the dependencies listed above.

Setup your PDP (Policy Decision Point) Container

To set up the PDP, you’ll need to start up docker. Once you do, open up your terminal and run this command:

docker pull permitio/pdp-v2:latest

After which you need to run the container with this command:

 docker run -it -p 7766:7000 --env PDP_DEBUG=True --env PDP_API_KEY=<YOUR_API_KEY> permitio/pdp-v2:latest

Replace the part that says <YOUR_API_KEY> with your actual API key. Now, let’s start building.

Build the Application

To build the application, this is how our project structure will be:


Bookstore                
├── config               
│   └── config.go        
│
├── handlers             
│   └── handlers.go      
│
├── middleware           
│   └── middleware.go    
│
├── models               
│   └── models.go        
│
├── templates            
│   ├── add.html         
│   ├── books.html       
│   ├── index.html       
│   ├── layout.html      
│   ├── login.html       
│   └── update.html      
│
├── main.go              
└── .env

Let's first add our API Key inside a .env file. Create one, and then your permit API key like so:

export PERMIT_API_KEY=”your_api_key”

Configure Database Connection

Create a folder called config. Inside it, create a file called config.go. Add the following code:


package config

import (
  "database/sql"
  "fmt"

  _ "github.com/lib/pq"
)

type Config struct {
  DB       *sql.DB
  Port     string
  DBConfig PostgresConfig
}

type PostgresConfig struct {
  Host     string
  Port     string
  User     string
  Password string
  DBName   string
}

func NewConfig() *Config {
  return &Config{
    Port: "8080",
    DBConfig: PostgresConfig{
      Host:     "localhost",
      Port:     "5432",
      User:     "bookstore_user",
      Password: "your_password",
      DBName:   "bookstore_db",
    },
  }
}

func (c *Config) ConnectDB() error {
  connStr := fmt.Sprintf("host=%s port=%s user=%s password=%s dbname=%s sslmode=disable",
    c.DBConfig.Host,
    c.DBConfig.Port,
    c.DBConfig.User,
    c.DBConfig.Password,
    c.DBConfig.DBName,
  )

  db, err := sql.Open("postgres", connStr)
  if err != nil {
    return fmt.Errorf("error opening database: %v", err)
  }

  if err := db.Ping(); err != nil {
    return fmt.Errorf("error connecting to database: %v", err)
  }

  c.DB = db
  return nil
}

This is just us setting up a configuration to connect to a PostgreSQL database.

Create the Handlers

Next, create a folder called handlers, and inside it, create a file called handlers.go. Inside it, add the following code:


package handlers

import (
  "bookstore/middleware"
  "bookstore/models" 
  "context"
  "database/sql"
  "fmt"
  "html/template"
  "net/http"
  "strings"
  "time"

  "github.com/google/uuid"
  "github.com/permitio/permit-golang/pkg/config"
  "github.com/permitio/permit-golang/pkg/enforcement"
  permitModels "github.com/permitio/permit-golang/pkg/models"
  "github.com/permitio/permit-golang/pkg/permit"
)

var tmpl = template.Must(template.ParseGlob("templates/*.html"))

func StringPtr(s string) *string {
  return &s
}

type Handlers struct {
  db           *sql.DB
  permitClient *permit.Client
}

func NewHandlers(db *sql.DB, apiKey string) *Handlers {
  permitConfig := config.NewConfigBuilder(apiKey).
    WithPdpUrl("http://localhost:7766").
    Build()
  permitClient := permit.NewPermit(permitConfig)
  if permitClient == nil {
    panic("Failed to initialize Permit.io client")
  }

  return &Handlers{
    db:           db,
    permitClient: permitClient,
  }
}

Aside from importing the packages, what we’re trying to do here is create a structure that holds the database connection and permit.io. We are also providing an initialization function that sets up Permit.io with local PDP.

Right after the NewHandlers add this in:


func (h *Handlers) LoginHandler() http.HandlerFunc {
        return func(w http.ResponseWriter, r *http.Request) {
                if r.Method == http.MethodGet {
                        if err := tmpl.ExecuteTemplate(w, "login.html", nil); err != nil {
                                http.Error(w, "Error rendering template", http.StatusInternalServerError)
                                return
                        }
                        return
                }

                username := r.FormValue("username")
                password := r.FormValue("password")

                user, err := middleware.LoginUser(h.db, username, password)
                if err != nil {
                        http.Error(w, "Invalid login credentials", http.StatusUnauthorized)
                        return
                }
                role := user.Role

                http.SetCookie(w, &http.Cookie{
                        Name:     "username",
                        Value:    username,
                        Path:     "/",
                        Expires:  time.Now().Add(24 * time.Hour),
                        HttpOnly: true,
                        Secure:   r.TLS != nil,
                })

                ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
                defer cancel()

                permitUser := permitModels.NewUserCreate(username)
                permitUser.SetAttributes(map[string]interface{}{
                        "role": role,
                })

                _, _ = h.permitClient.SyncUser(ctx, *permitUser)

                data := struct {
                        Username string
                        Role     string
                }{
                        Username: username,
                        Role:     role,
                }

                if err := tmpl.ExecuteTemplate(w, "index.html", data); err != nil {
                        http.Error(w, "Error displaying page", http.StatusInternalServerError)
                }
        }
}

The LoginHandler does the following:

Handles both GET (show login form) and POST (process login).
Authenticates users against the database.
Sets session cookies for authenticated users.
Syncs user data with Permit.io for authorization.
Renders appropriate templates based on login success/failure.

The next step is to add a book handler to access the books. It will also utilize permit.io to verify the user's role. Add the following code right after the LoginHandler:


func (h *Handlers) BooksHandler() http.HandlerFunc {
        return func(w http.ResponseWriter, r *http.Request) {
                cookie, err := r.Cookie("username")
                if err != nil {
                        http.Error(w, "Unauthorized access: no username found", http.StatusUnauthorized)
                        return
                }
                username := cookie.Value

                role, err := middleware.GetUserRole(h.db, username)
                if err != nil {
                        http.Error(w, "Error retrieving user role", http.StatusInternalServerError)
                        return
                }

                user := enforcement.UserBuilder(username).
                        WithAttributes(map[string]interface{}{
                                "role": role,
                        }).
                        Build()

                resource := enforcement.ResourceBuilder("books").
                        WithTenant("default").
                        Build()

                permitted, err := h.permitClient.Check(user, "view", resource)
                if err != nil {
                        http.Error(w, "Error checking permissions", http.StatusInternalServerError)
                        return
                }

                if !permitted {
                        http.Error(w, "Access denied", http.StatusForbidden)
                        return
                }

                rows, err := h.db.Query("SELECT id, title, author, published_at, created_at FROM books")
                if err != nil {
                        http.Error(w, "Error fetching books", http.StatusInternalServerError)
                        return
                }
                defer rows.Close()

                var books []models.Book
                for rows.Next() {
                        var book models.Book
                        var publishedAt sql.NullTime

                        err := rows.Scan(&book.ID, &book.Title, &book.Author, &publishedAt, &book.CreatedAt)
                        if err != nil {
                                http.Error(w, "Error reading book data", http.StatusInternalServerError)
                                return
                        }

                        if publishedAt.Valid {
                                book.PublishedAt = &publishedAt.Time
                        }

                        books = append(books, book)
                }

                if err = rows.Err(); err != nil {
                        http.Error(w, "Error reading book data", http.StatusInternalServerError)
                        return
                }

                if err := tmpl.ExecuteTemplate(w, "books.html", books); err != nil {
                        http.Error(w, "Error displaying books", http.StatusInternalServerError)
                }
        }
}

The BookHandler does the following:

Checks user authentication via cookies.
Verifies user role and permissions using Permit.io.
Fetches books from the database if authorized.
Renders books template with fetched data.
Handles authorization failures appropriately.

Next, you need a handler to add books. It will also verify the user's role through Permit.io to ensure only authorized users can add books:


func (h *Handlers) AddBookHandler() http.HandlerFunc {
        return func(w http.ResponseWriter, r *http.Request) {
                cookie, err := r.Cookie("username")
                if err != nil {
                        http.Error(w, "Unauthorized access: no username found", http.StatusUnauthorized)
                        return
                }
                username := cookie.Value

                role, err := middleware.GetUserRole(h.db, username)
                if err != nil {
        func (h *Handlers) AddBookHandler() http.HandlerFunc {
        return func(w http.ResponseWriter, r *http.Request) {
                cookie, err := r.Cookie("username")
                if err != nil {
                        http.Error(w, "Unauthorized access: no username found", http.StatusUnauthorized)
                        return
                }
                username := cookie.Value

                role, err := middleware.GetUserRole(h.db, username)
                if err != nil {
                        http.Error(w, "Error retrieving user role", http.StatusInternalServerError)
                        return
                }

                user := enforcement.UserBuilder(username).
                        WithAttributes(map[string]interface{}{
                                "role": role,
                        }).
                        Build()

                resource := enforcement.ResourceBuilder("books").
                        WithTenant("default").
                        Build()

                permitted, err := h.permitClient.Check(user, "create", resource)
                if err != nil {
                        http.Error(w, "Error checking permissions", http.StatusInternalServerError)
                        return
                }

                if !permitted {
                        w.Header().Set("Content-Type", "text/html")
                        fmt.Fprint(w, `
                                <!DOCTYPE html>
                                <html lang="en">
                                <head>
                                        <meta charset="UTF-8">
                                        <meta name="viewport" content="width=device-width, initial-scale=1.0">
                                        <title>Access Denied</title>
                                        <script>alert('You cannot add a book as you are an admin.')</script>
                                </head>
                                <body>
                                        <p>You do not have permission to add books.</p>
                                        <a href="/books">Back to Books</a>
                                </body>
                                </html>
                        `)
                        return
                }

                if r.Method == http.MethodGet {
                        if err := tmpl.ExecuteTemplate(w, "add.html", nil); err != nil {
                                http.Error(w, "Error displaying page", http.StatusInternalServerError)
                        }
                        return
                }

                if r.Method == http.MethodPost {
                        title := strings.TrimSpace(r.FormValue("title"))
                        author := strings.TrimSpace(r.FormValue("author"))
                        publishedAt := strings.TrimSpace(r.FormValue("published_at"))

                        var pubDate sql.NullTime
                        if publishedAt != "" {
                                parsedDate, err := time.Parse("2006-01-02", publishedAt)
                                if err == nil {
                                        pubDate = sql.NullTime{Time: parsedDate, Valid: true}
                                }
                        }

                        _, err := h.db.Exec("INSERT INTO books (id, title, author, published_at, created_at) VALUES ($1, $2, $3, $4, NOW())",
                                uuid.New(), title, author, pubDate)
                        if err != nil {
                                http.Error(w, "Error adding book", http.StatusInternalServerError)
                                return
                        }

                        http.Redirect(w, r, "/books", http.StatusSeeOther)
                }
        }
}

The AddBookHandler does the following:

Checks user permissions for book creation.
Handles both GET (show form) and POST (add book).
Validates input data.
Generates UUID for new books.
Handles date parsing for publication dates.
Redirects to books list after successful addition.

You need two more handlers, one for deleting and the other for updating. Add this code right after the AddBookHandler function:


func (h *Handlers) DeleteBookHandler() http.HandlerFunc {
        return func(w http.ResponseWriter, r *http.Request) {
                cookie, err := r.Cookie("username")
                if err != nil {
                        http.Error(w, "Unauthorized access: no username found", http.StatusUnauthorized)
                        return
                }
                username := cookie.Value

                role, err := middleware.GetUserRole(h.db, username)
                if err != nil {
                        http.Error(w, "Error retrieving user role", http.StatusInternalServerError)
                        return
                }

                user := enforcement.UserBuilder(username).
                        WithAttributes(map[string]interface{}{
                                "role": role,
                        }).
                        Build()

                resource := enforcement.ResourceBuilder("books").
                        WithTenant("default").
                        Build()

                permitted, err := h.permitClient.Check(user, "delete", resource)
                if err != nil {
                        http.Error(w, "Error checking permissions", http.StatusInternalServerError)
                        return
                }

                if !permitted {
                        http.Error(w, "Access denied", http.StatusForbidden)
                        return
                }

                bookIDStr := r.FormValue("id")
                bookID, err := uuid.Parse(bookIDStr)
                if err != nil {
                        http.Error(w, "Invalid book ID", http.StatusBadRequest)
                        return
                }

                _, err = h.db.Exec("DELETE FROM books WHERE id = $1", bookID)
                if err != nil {
                        http.Error(w, "Error deleting book", http.StatusInternalServerError)
                        return
                }

                http.Redirect(w, r, "/books", http.StatusSeeOther)
        }
}

The DeleteBookHandler does the following:

Verifies user permissions for deletion.
Validates book ID.
Performs database deletion.
Handles errors and redirects appropriately.

Right after the DeleteBookHandler function, add the following:


func (h *Handlers) UpdateBookHandler() http.HandlerFunc {
        return func(w http.ResponseWriter, r *http.Request) {
                cookie, err := r.Cookie("username")
                if err != nil {
                        http.Error(w, "Unauthorized access: no username found", http.StatusUnauthorized)
                        return
                }
                username := cookie.Value

                role, err := middleware.GetUserRole(h.db, username)
                if err != nil {
                        http.Error(w, "Error retrieving user role", http.StatusInternalServerError)
                        return
                }

                user := enforcement.UserBuilder(username).
                        WithAttributes(map[string]interface{}{
                                "role": role,
                        }).
                        Build()

                resource := enforcement.ResourceBuilder("books").
                        WithTenant("default").
                        Build()

                permitted, err := h.permitClient.Check(user, "update", resource)
                if err != nil {
                        http.Error(w, "Error checking permissions", http.StatusInternalServerError)
                        return
                }

                if !permitted {
                        w.Header().Set("Content-Type", "text/html")
                        fmt.Fprint(w, `
                                <!DOCTYPE html>
                                <html lang="en">
                                <head>
                                        <meta charset="UTF-8">
                                        <meta name="viewport" content="width=device-width, initial-scale=1.0">
                                        <title>Access Denied</title>
                                        <script>alert('You are not authorized to update books.')</script>
                                </head>
                                <body>
                                        <p>Access Denied. You do not have permission to update books.</p>
                                        <a href="/books">Back to Books</a>
                                </body>
                                </html>
                        `)
                        return
                }

                if r.Method == http.MethodGet {
                        bookID := r.FormValue("id")
                        var book models.Book
                        err := h.db.QueryRow("SELECT id, title, author, published_at FROM books WHERE id = $1", bookID).
                                Scan(&book.ID, &book.Title, &book.Author, &book.PublishedAt)
                        if err != nil {
                                http.Error(w, "Error fetching book", http.StatusInternalServerError)
                                return
                        }

                        if err := tmpl.ExecuteTemplate(w, "update.html", book); err != nil {
                                http.Error(w, "Error displaying update page", http.StatusInternalServerError)
                        }
                        return
                }

                if r.Method == http.MethodPost {
                        bookID := r.FormValue("id")
                        title := r.FormValue("title")
                        author := r.FormValue("author")
                        publishedAt := r.FormValue("published_at")

                        var pubDate sql.NullTime
                        if publishedAt != "" {
                                parsedDate, err := time.Parse("2006-01-02", publishedAt)
                                if err == nil {
                                        pubDate = sql.NullTime{Time: parsedDate, Valid: true}
                                }
                        }

                        _, err := h.db.Exec("UPDATE books SET title = $1, author = $2, published_at = $3 WHERE id = $4",
                                title, author, pubDate, bookID)
                        if err != nil {
                                http.Error(w, "Error updating book", http.StatusInternalServerError)
                                return
                        }

                        http.Redirect(w, r, "/books", http.StatusSeeOther)
                }
        }
}

The UpdateHandler does the following:

Checks update permissions.
Handles both GET (show edit form) and POST (update book).
Fetches existing book data for editing.
Validates and processes updates.
Handles date formatting and database updates.

Throughout the code, you’ll notice that the authorization system is built around Permit.io's role-based access control framework, which provides sophisticated permission management.

This system also enables fine-grained control over user actions and allows different levels of access for viewing, creating, updating, and deleting resources. Each operation in the application undergoes detailed permission checking and ensures that users can only perform actions for which they're authorized.

Create Authorization Middleware

Now we’re done with the handlers. Create a folder called middleware, and inside it, create a file called middleware.go. Add the following code:


package middleware

import (
        "bookstore/models"
        "database/sql"
        "fmt"
        "github.com/google/uuid"
        "golang.org/x/crypto/bcrypt"
        "time"
)

func HashPassword(password string) (string, error) {
        bytes, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
        return string(bytes), err
}

func CheckPasswordHash(password, hash string) bool {
        err := bcrypt.CompareHashAndPassword([]byte(hash), []byte(password))
        return err == nil
}

func LoginUser(db *sql.DB, username, password string) (*models.User, error) {
        fmt.Printf("Attempting login for username: %s\n", username)

        var user models.User
        var passwordHash string

        err := db.QueryRow(`
       SELECT id, username, password_hash, role, email, first_name, last_name, created_at
       FROM users
       WHERE username = $1
   `, username).Scan(
                &user.ID,
                &user.Username,
                &passwordHash,
                &user.Role,
                &user.Email,
                &user.FirstName,
                &user.LastName,
                &user.CreatedAt,
        )

        if err != nil {
                if err == sql.ErrNoRows {
                        fmt.Printf("No user found with username: %s\n", username)
                        return nil, fmt.Errorf("invalid credentials")
                }
                fmt.Printf("Database error: %v\n", err)
                return nil, err
        }

        if err := bcrypt.CompareHashAndPassword([]byte(passwordHash), []byte(password)); err != nil {
                fmt.Printf("Password comparison failed: %v\n", err)
                return nil, fmt.Errorf("invalid credentials")
        }

        user.PasswordHash = ""
        return &user, nil
}

func GetBooks(db *sql.DB) ([]models.Book, error) {
        rows, err := db.Query(`
       SELECT id, title, author, published_at, created_by, created_at
       FROM books
       ORDER BY created_at DESC
   `)
        if err != nil {
                return nil, err
        }
        defer rows.Close()

        var books []models.Book
        for rows.Next() {
                var book models.Book
                err := rows.Scan(
                        &book.ID,
                        &book.Title,
                        &book.Author,
                        &book.PublishedAt,
                        &book.CreatedAt,
                )
                if err != nil {
                        return nil, err
                }
                books = append(books, book)
        }

        return books, nil
}

func CreateBook(db *sql.DB, book *models.Book) error {
        book.ID = uuid.New()
        book.CreatedAt = time.Now()

        _, err := db.Exec(`
       INSERT INTO books (id, title, author, published_at, created_by, created_at)
       VALUES ($1, $2, $3, $4, $5, $6)
   `,
                book.ID,
                book.Title,
                book.Author,
                book.PublishedAt,
                book.CreatedAt,
        )

        return err
}

func UpdateBook(db *sql.DB, book *models.Book) error {
        result, err := db.Exec(`
       UPDATE books
       SET title = $1, author = $2, published_at = $3
       WHERE id = $4 AND created_by = $5
   `,
                book.Title,
                book.Author,
                book.PublishedAt,
                book.ID,
        )
        if err != nil {
                return err
        }

        rowsAffected, err := result.RowsAffected()
        if err != nil {
                return err
        }

        if rowsAffected == 0 {
                return fmt.Errorf("book not found or user not authorized")
        }

        return nil
}

func DeleteBook(db *sql.DB, bookID uuid.UUID, userID uuid.UUID) error {
        result, err := db.Exec(`
       DELETE FROM books
       WHERE id = $1 AND created_by = $2
   `, bookID, userID)
        if err != nil {
                return err
        }

        rowsAffected, err := result.RowsAffected()
        if err != nil {
                return err
        }

        if rowsAffected == 0 {
                return fmt.Errorf("book not found or user not authorized")
        }

        return nil
}

func GetUserRole(db *sql.DB, username string) (string, error) {
        var role string
        err := db.QueryRow("SELECT role FROM users WHERE username = $1", username).Scan(&role)
        if err != nil {
                if err == sql.ErrNoRows {
                        return "", fmt.Errorf("no user found with username %s", username)
                }
                return "", err
        }
        return role, nil
}

This middleware package helps provide secure password hashing and authentication, along with CRUD operations for managing books in a bookstore application. It uses bcrypt to hash passwords for secure storage and verifies password hashes during login. It also prevents the exposure of sensitive data.

The LoginUser function authenticates users by comparing their input with stored password hashes and retrieves the full user profile on successful login, excluding the password hash for added security.

Also, CRUD operations allow you to create, update, retrieve, and delete book records in the database, with access control to ensure only authorized users can modify or delete entries they created. The package also includes a GetUserRole function to retrieve user roles, facilitating role-based access control.

Create the Models

Create another folder called models, and inside it, create a file called models.go. And add the following:


package models

import (
        "time"

        "database/sql/driver"

        "github.com/google/uuid"
)

type NullUUID struct {
        UUID  uuid.UUID
        Valid bool
}

func (n *NullUUID) Scan(value interface{}) error {
        if value == nil {
                n.UUID, n.Valid = uuid.UUID{}, false
                return nil
        }
        n.Valid = true
        return n.UUID.Scan(value)
}

func (n NullUUID) Value() (driver.Value, error) {
        if !n.Valid {
                return nil, nil
        }
        return n.UUID[:], nil
}

type User struct {
        ID           uuid.UUID `json:"id"`
        Username     string    `json:"username"`
        PasswordHash string    `json:"-"`
        Role         string    `json:"role"`
        Email        string    `json:"email"`
        FirstName    string    `json:"first_name"`
        LastName     string    `json:"last_name"`
        CreatedAt    time.Time `json:"created_at"`
}

type Book struct {
        ID          uuid.UUID  `json:"id"`
        Title       string     `json:"title"`
        Author      string     `json:"author"`
        PublishedAt *time.Time `json:"published_at,omitempty"`

        CreatedAt time.Time `json:"created_at"`
}

type LoginRequest struct {
        Username string `json:"username"`
        Password string `json:"password"`
}

This package defines several data models for a bookstore application, including User, Book, and LoginRequest structures, along with a custom NullUUID type for handling nullable UUIDs in the database.

Almost done. The next thing you need to do is create the templates for your project. You’ll need to create templates for login and index, to add books, view books, delete books, and update books.

Create the HTML Templates

Create a folder called templates. This is where your html templates will be.
For login, create a file called login.html, and inside it, paste this:


    <!doctype html>
    <html lang="en">
      <head>
        <meta charset="UTF-8" />
        <title>Login</title>
      </head>
      <body>
        <h2>Login</h2>
        <form method="POST" action="/login">
          <label for="username">Username:</label>
          <input type="text" id="username" name="username" required /><br />
          <label for="password">Password:</label>
          <input type="password" id="password" name="password" required /><br />
          <button type="submit">Login</button>
        </form>
      </body>
    </html>

For index, create a file called `index.html`. Add this:


    <!doctype html>
    <html>
      <head>
        <title>Welcome</title>
      </head>
      <body>
        <h1>Welcome {{.Username}}!</h1>
        <p>You are logged in successfully.</p>
        <p>Your role is: {{.Role}}</p>
        <a href="/books">Go to Books</a>
        <br />
        <a href="/add">Add Book</a>
      </body>
    </html>

For viewing books, create a file called books.html. Add this:


<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Books</title>
    <link
      rel="stylesheet"
      href="https://cdn.jsdelivr.net/npm/tailwindcss@2.2.19/dist/tailwind.min.css"
    />
  </head>
  <body class="bg-gray-100">
    <div class="container mx-auto px-4">
      <h1 class="text-3xl font-bold text-center my-8">Books</h1>
      <div class="text-center mb-4">
        <a
          href="/add"
          class="bg-blue-500 text-white px-4 py-2 rounded hover:bg-blue-600"
          >Add Book</a
        >
      </div>

      {{if eq (len .) 0}}
      <p class="text-center text-gray-600">No books to fetch</p>
      {{else}}
      <div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-6">
        {{range .}}
        <div class="bg-white shadow-md rounded-lg p-6 relative">
          <h2 class="text-xl font-bold mb-2">{{.Title}}</h2>
          <p><strong>Author:</strong> {{.Author}}</p>
          {{if .PublishedAt}}
          <p>
            <strong>Published Date:</strong> {{.PublishedAt.Format
            "2006-01-02"}}
          </p>
          {{else}}
          <p><strong>Published Date:</strong> Unknown</p>
          {{end}}
          <p><strong>Created At:</strong> {{.CreatedAt.Format "2006-01-02"}}</p>

          <!-- Update and Delete Buttons -->
          <div class="mt-4 flex space-x-2">
            <form action="/update" method="GET">
              <input type="hidden" name="id" value="{{.ID}}" />
              <button
                type="submit"
                class="bg-yellow-500 text-white px-4 py-2 rounded hover:bg-yellow-600 focus:outline-none"
              >
                Update
              </button>
            </form>
            <form action="/delete" method="POST">
              <input type="hidden" name="id" value="{{.ID}}" />
              <button
                type="submit"
                class="bg-red-500 text-white px-4 py-2 rounded hover:bg-red-600 focus:outline-none"
              >
                Delete
              </button>
            </form>
          </div>
        </div>
        {{end}}
      </div>
      {{end}}
    </div>
  </body>
</html>

To add books, create a file called add.html. Add this:


<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Add a New Book</title>
    <link
      rel="stylesheet"
      href="https://cdn.jsdelivr.net/npm/tailwindcss@2.2.19/dist/tailwind.min.css"
    />
  </head>
  <body class="bg-gray-100">
    <div class="max-w-md mx-auto bg-white rounded-lg shadow-md p-6 mt-10">
      <h2 class="text-2xl font-bold mb-6">Add a New Book</h2>

      <!-- Form to add a new book -->
      <form action="/add" method="POST">
        <div class="mb-4">
          <label class="block text-gray-700 text-sm font-bold mb-2" for="title"
            >Title</label
          >
          <input
            type="text"
            id="title"
            name="title"
            class="shadow appearance-none border rounded w-full py-2 px-3 text-gray-700 leading-tight focus:outline-none focus:shadow-outline"
            required
          />
        </div>

        <div class="mb-4">
          <label class="block text-gray-700 text-sm font-bold mb-2" for="author"
            >Author</label
          >
          <input
            type="text"
            id="author"
            name="author"
            class="shadow appearance-none border rounded w-full py-2 px-3 text-gray-700 leading-tight focus:outline-none focus:shadow-outline"
            required
          />
        </div>

        <div class="mb-6">
          <label
            class="block text-gray-700 text-sm font-bold mb-2"
            for="published_at"
            >Published Date</label
          >
          <input
            type="date"
            id="published_at"
            name="published_at"
            class="shadow appearance-none border rounded w-full py-2 px-3 text-gray-700 leading-tight focus:outline-none focus:shadow-outline"
          />
        </div>

        <div class="flex items-center justify-between">
          <button
            type="submit"
            class="bg-indigo-600 text-white px-4 py-2 rounded-md hover:bg-indigo-700 focus:outline-none focus:shadow-outline"
          >
            Add Book
          </button>
        </div>
      </form>

      <!-- Link back to books page -->
      <div class="mt-4">
        <a href="/books" class="text-indigo-600 hover:underline"
          >Back to Books</a
        >
      </div>
    </div>
  </body>
</html>

Lastly, you need to create the main file, which is the main.go. This will be in the root folder. Create it and add the following code:


package main

import (
        "bookstore/handlers"
        "database/sql"
        "fmt"
        "log"
        "net/http"
        "os"

        "github.com/gorilla/mux"
        "github.com/joho/godotenv"
        _ "github.com/lib/pq"
)

func connectDB() *sql.DB {

        connStr := "user=bookstore_user password=1111 dbname=bookstore_db sslmode=disable"
        db, err := sql.Open("postgres", connStr)
        if err != nil {
                log.Fatal("Error connecting to the database:", err)
        }
        if err := db.Ping(); err != nil {
                log.Fatal("Cannot ping the database:", err)
        }
        fmt.Println("Successfully connected to the database!")
        return db
}

func main() {

        err := godotenv.Load()
        if err != nil {
                log.Print("Error loading .env file")
        }

        permitApiKey := os.Getenv("PERMIT_API_KEY")
        if permitApiKey == "" {
                log.Fatal("PERMIT_API_KEY environment variable is not set")
        }

        db := connectDB()
        defer db.Close()

        r := mux.NewRouter()

        h := handlers.NewHandlers(db, permitApiKey)

        r.HandleFunc("/login", h.LoginHandler()).Methods("GET", "POST")
        r.HandleFunc("/books", h.BooksHandler()).Methods("GET")
        r.HandleFunc("/add", h.AddBookHandler()).Methods("GET", "POST")
        r.HandleFunc("/delete", h.DeleteBookHandler()).Methods("POST")
        r.HandleFunc("/update", h.UpdateBookHandler()).Methods("GET", "POST")

        fmt.Println("Server starting on :8080")
        if err := http.ListenAndServe(":8080", r); err != nil {
                log. Fatal(err)
        }
}

This main package serves as the entry point for a bookstore application. It sets up database connectivity, environment configuration, and HTTP routes for handling user login and book management.

In the main function, routes are registered using the Gorilla Mux router. The handlers.NewHandlers function initializes handlers with the database and Permit.io API key. It enables functionality such as user authentication (/login) and book management (/books, /add, /delete, /update). Each route is mapped to specific HTTP methods, organizing the endpoints for different actions.

Finally, the server starts on port 8080, listening for incoming requests and logging any errors that occur. This setup ensures a structured API endpoint configuration and secure handling of environment variables.

Test the Application

Now that's about everything! Let's start up our app to see the result. To start the server, run this command:

go run main.go

Visit http://localhost:8080/login in your browser.

Let's start by testing just the permissions of the standard_user:

You'll see that the standard_user is restricted to viewing books only and cannot add, delete, or update a book.

Let's now log in using the admin_user to see what happens:

You'll see that the admin has permission to do just about anything! That's how solid and easy to use Permit is!

You can check out these resources to learn more about Permit’s authorization:

Conclusion

In this tutorial, we built a simple bookstore management app to implement role-based access control using Go, HTMX, and Permit.io. Authorization is a fundamental aspect of application security, as it ensures that users can only access what they’re allowed to.

Implementing an effective access control model like RBAC or ABAC into your application would not only secure your application but also enhance its scalability and compliance.

Understanding the React Cache function

Oyedele Temitope — Thu, 12 Sep 2024 10:06:04 +0000

With React's ecosystem expanding, one of the more powerful tools for optimizing data fetching is the cache function. This built-in feature allows you to do a lot of things like manage and store server data effectively, reduce redundant network requests and also improve overall app performance.

In this article, we'll look at the cache function in React, its benefits, and how to use it.

What is React `cache` Function

The cache function released by React is designed to optimize performance. It does so by avoiding unnecessary computations when the same arguments are passed to a function. This is possible through a mechanism known as memoization, where the results of function calls are stored and reused if the same inputs occur again.

React's cache function helps prevent a function from being executed repeatedly with the same arguments, thus saving computational resources and improving the overall efficiency of the application.

To use the cache function, you wrap the target function with cache, and React takes care of storing the results of the function calls. When the wrapped function is called again with the same arguments, React checks the cache first. If the result for those arguments exists in the cache, it returns the cached result instead of executing the function again.

This behavior ensures that the function only runs when necessary, i.e., when the arguments are different from those previously seen.

Here's a simple example demonstrating how you can use React's cache function to skip duplicate work when fetching data from a weather application:

import { cache } from 'react';
import { Suspense } from 'react';

const fetchWeatherData = async (city) => {
  console.log(`Fetching weather data for ${city}...`);
  // Simulate API call
  await new Promise(resolve => setTimeout(resolve, 2000));
  return { 
    temperature: Math.round(Math.random() * 30),
    conditions: ['Sunny', 'Cloudy', 'Rainy'][Math.floor(Math.random() * 3)]
  };
};

const getCachedWeatherData = cache(fetchWeatherData);

async function WeatherWidget({ city }) {
  const weatherData = await getCachedWeatherData(city);
  return (
    <div>
      <h2>Weather in {city}</h2>
      <p>Temperature: {weatherData.temperature}°C</p>
      <p>Conditions: {weatherData.conditions}</p>
    </div>
  );
}

function WeatherDashboard() {
  return (
    <div>
      <Suspense fallback={<div>Loading New York weather...</div>}>
        <WeatherWidget city="New York" />
      </Suspense>
      <Suspense fallback={<div>Loading London weather...</div>}>
        <WeatherWidget city="London" />
      </Suspense>
      <Suspense fallback={<div>Loading New York weather...</div>}>
        <WeatherWidget city="New York" /> {/* Duplicate */}
      </Suspense>
      <Suspense fallback={<div>Loading Tokyo weather...</div>}>
        <WeatherWidget city="Tokyo" />
      </Suspense>
    </div>
  );
}

export default WeatherDashboard;

In the code above, the cache function is applied to fetchWeatherData, creating a new function getCachedWeatherData that memorizes the results of weather data fetches. This cached function is then used within the WeatherWidget component to retrieve weather information for different cities.

The WeatherDashboard component renders multiple instances of WeatherWidget and includes a duplicate for New York, which is deliberate. This serves as a crucial proof of concept for the caching mechanism, as it prevents redundant, expensive operations when the same data is requested multiple times within a render cycle by reusing the cached result from the first call, avoiding an unnecessary network request.

This caching mechanism has several advantages, one of which is that it reduces the number of API calls you make to the server and this can greatly improve the performance and lower server load. It also ensures data consistency across components requesting the same information, and it simplifies component code by automatically handling potential duplicate requests.

It's important to note that React's cache function is intended for use in Server Components only. Each call to cache creates a new memoized function, meaning that calling cache multiple times with the same function will result in separate memoized versions that do not share the same cache.

Another thing to note is that the cache function caches both successful results and errors. So, if a function throws an error for certain arguments, that error will be cached and re-thrown upon subsequent calls with those same arguments.

Benefits of the `cache` Function

The benefits of the cache function are numerous. Below are some of its benefits:

Improved Application Performance: Reusing cached data across components reduces server requests and leads to faster response times and a smoother user experience.
Efficient Data Fetching: It minimizes redundant data-fetching operations, especially in server-side rendering or static generation. This can help save time and resources when frequently requested or costly data is involved.
Reduced Server Load:It helps reduce the load on servers, thereby improving scalability and backend reliability.
Enhanced User Experience: Faster loading times and reduced latency allow users to interact with the application easily.
Integration with Caching Strategies: The cache function complements tools like useMemo and useCallback, offering a robust approach to optimizing performance.

When to Use the Cache Function

You can use the cache function when you want to:

Prevent Redundant API Calls: If you’re fetching data from an API or database and the same function with the same arguments is called multiple times, cache ensures the request is made only once.
Optimize Expensive Computations: it can be used for resource-intensive operations that take significant time or server resources, cache can store the results, so you don’t need to recompute them for the same inputs.
Share Data Across Components: When different components in your application require the same data, the cache ensures that the data-fetching logic is reused, and this helps you avoid doing repeated work, thus, improving server efficiency.
Preload Data: You can leverage cache to preload data before a component renders. This is particularly useful for critical data that needs to be available immediately on the initial render.

Conclusion

The cache function in Next.js, combined with React's built-in caching capabilities, offers a powerful toolkit for optimizing data fetching and component rendering in your application. Caching data can help improve performance, reduce unnecessary API calls, and enhance the user experience.

Remember, React's cache function is an experimental feature and subject to change. Always refer to the latest React documentation for the most current information and usage guidelines.

Understanding the React Cache function

Oyedele Temitope — Thu, 12 Sep 2024 10:06:03 +0000

With React's ecosystem expanding, one of the more powerful tools for optimizing data fetching is the cache function. This built-in feature allows you to do a lot of things like manage and store server data effectively, reduce redundant network requests and also improve overall app performance.

In this article, we'll look at the cache function in React, its benefits, and how to use it.

What is React `cache` Function

React's cache function helps prevent a function from being executed repeatedly with the same arguments, thus saving computational resources and improving the overall efficiency of the application.

This behavior ensures that the function only runs when necessary, i.e., when the arguments are different from those previously seen.

Here's a simple example demonstrating how to use React's cache function to skip duplicate work when fetching data from a weather application:

import { cache } from 'react';
import { Suspense } from 'react';

const fetchWeatherData = async (city) => {
  console.log(`Fetching weather data for ${city}...`);
  // Simulate API call
  await new Promise(resolve => setTimeout(resolve, 2000));
  return { 
    temperature: Math.round(Math.random() * 30),
    conditions: ['Sunny', 'Cloudy', 'Rainy'][Math.floor(Math.random() * 3)]
  };
};

const getCachedWeatherData = cache(fetchWeatherData);

async function WeatherWidget({ city }) {
  const weatherData = await getCachedWeatherData(city);
  return (
    <div>
      <h2>Weather in {city}</h2>
      <p>Temperature: {weatherData.temperature}°C</p>
      <p>Conditions: {weatherData.conditions}</p>
    </div>
  );
}

function WeatherDashboard() {
  return (
    <div>
      <Suspense fallback={<div>Loading New York weather...</div>}>
        <WeatherWidget city="New York" />
      </Suspense>
      <Suspense fallback={<div>Loading London weather...</div>}>
        <WeatherWidget city="London" />
      </Suspense>
      <Suspense fallback={<div>Loading New York weather...</div>}>
        <WeatherWidget city="New York" /> {/* Duplicate */}
      </Suspense>
      <Suspense fallback={<div>Loading Tokyo weather...</div>}>
        <WeatherWidget city="Tokyo" />
      </Suspense>
    </div>
  );
}

export default WeatherDashboard;

In the code above, the cache function is applied to fetchWeatherData, creating a new function getCachedWeatherData that memorizes the results of weather data fetches. This cached function is then used within the WeatherWidget component to retrieve weather information for different cities.

The WeatherDashboard component renders multiple instances of WeatherWidget, including a duplicate for New York, which is deliberate. This serves as a crucial proof of concept for the caching mechanism, as it prevents redundant expensive operations when the same data is requested multiple times within a render cycle by reusing the cached result from the first call, avoiding an unnecessary network request.

This caching mechanism has several advantages: it reduces the number of API calls, resulting in improved performance and lower server load; it ensures data consistency across components requesting the same information; and it simplifies component code by automatically handling potential duplicate requests.

This feature is part of React's broader strategy to enhance performance and efficiency, complementing existing mechanisms like the virtual DOM and the useMemo and useCallback hooks, which also employ memoization techniques to optimize component rendering and function references.

Benefit of the `cache` Function

The benefits of using React's cache function primarily revolve around performance optimization, specifically in terms of reducing unnecessary computations and data fetching operations. Below are some key benefits of the cache function:

Improved Application Performance: Caching helps in reducing the number of server requests needed by reusing cached data across multiple components. This leads to faster response times and a smoother user experience, as the application spends less time waiting for data to be fetched or computed.
Efficient Data Fetching: In scenarios involving data fetching, especially in server-side rendering or static generation contexts, caching can significantly reduce the amount of data that needs to be fetched from the server. This is particularly beneficial in applications where the same data is requested frequently or where data fetching is costly in terms of performance.
Reduced Load on Servers: By serving data from the cache instead of making new requests to the server, caching helps in distributing the load more evenly. This can lead to better scalability and reliability of backend services, as they are not overwhelmed by frequent identical requests.
Enhanced User Experience: Faster loading times and reduced latency contribute to a better user experience. Users can interact with the application more quickly, as the application spends less time fetching or computing data.
Support for Advanced Caching Strategies: React's cache function complements other caching mechanisms and strategies, such as memoization (useMemo) and callback memoization (useCallback). These tools together offer a comprehensive approach to optimizing React applications, allowing developers to fine-tune performance based on specific needs.

When to Use the Cache Function

You can use the cache function when you want to :

Memoize Expensive Data Fetches: If your Server Component relies on fetching data from an API or performing complex calculations, wrapping the data fetching function with cache can significantly improve performance. The function will only be executed once for the same arguments, and subsequent renders will use the cached result.
Preload Data: You can leverage cache to preload data before a component even renders. This is particularly useful for critical data that needs to be available immediately on the initial render.
Share Results Across Components: When multiple Server Components require the same data fetched from the server, using cache ensures a single request is made, and the result is shared across all components, reducing redundant server calls.

Conclusion

The cache function in Next.js, combined with React's built-in caching capabilities, offers a powerful toolkit for optimizing data fetching and component rendering in your application. By strategically caching data and computations, you can significantly improve performance, reduce unnecessary API calls, and enhance the user experience.

Remember, React's cache function is an experimental feature and subject to change. Always refer to the latest React documentation for the most current information and usage guidelines.

Understanding the Child Process Module in Node.js

Oyedele Temitope — Mon, 09 Sep 2024 11:21:11 +0000

The child process module is significant in Node.js applications as it helps facilitate parallel processing and allows you to offload resource-intensive tasks to separate processes.

In this article, we'll take a look at the child process module, explaining its purpose, use cases, and how to create it.

What is the child process module

The child process is a core module that allows users to create and control subprocesses. These processes can execute system commands, run scripts in various languages, or even fork new instances of Node.js.

The primary purpose of the child process module is to allow the execution of multiple processes simultaneously without blocking the main event loop.

Using a child process is important for applications that need to handle CPU-intensive operations or execute external commands and scripts. With it, your applications can maintain high performance and responsiveness.

Use cases for child processes

The child processes can be used for tasks including the following:

Parallel processing: Child processing enables parallel processing by allowing your applications to distribute workloads across multiple CPU cores, considerably improving performance for CPU-intensive activities such as image processing and data analysis.
Running shell scripts: Child processes can be used to execute shell scripts. You can use the exec technique to run shell commands and capture their output, and also the spawn method which offers greater control when directly running scripts.
Communication with other services: The child processes play a vital role when it comes to communication. It communicates with external services such as databases, APIs, or microservices. They can be used to make calls to external APIs, do database queries, and communicate with other microservices.

Creating a child process

To create a child process, Node.js provides us with four primary methods for creating a child process which are the exec(),execFile(),spawn(),fork()

The `exec()` method

The exec() method runs a command in a shell and buffers the output. It is handy for running shell commands and recording the output. However, due to buffering, it has a memory limit.

Below is an example of using exec() method to execute a command that fetches and processes some system information.



const { exec } = require("child_process");

exec("df -h", (error, stdout, stderr) => {
  if (error) {
    console.error(`exec error: ${error.message}`);
    return;
  }
  if (stderr) {
    console.error(`stderr: ${stderr}`);
    return;
  }

  const lines = stdout.trim().split("\n");
  const diskInfo = lines[1].split(/\s+/);

  const totalSpace = diskInfo[1];
  const usedSpace = diskInfo[2];
  const availableSpace = diskInfo[3];
  const usagePercent = diskInfo[4];

  console.log(`Total Space: ${totalSpace}`);
  console.log(`Used Space: ${usedSpace}`);
  console.log(`Available Space: ${availableSpace}`);
  console.log(`Usage: ${usagePercent}`);
});

This will execute the df -h command, parse its output, and display the total, used, and available disk space along with the usage percentage.

When you run the code, you should see something like this:

The `execFile()` method

The execFile() method runs an executable file without a shell. It’s more efficient than exec() for running binaries and scripts directly since it avoids the overhead of a shell.

Below is an example of how to use the execFile() method:



const { execFile } = require('child_process');
execFile("node", ["--version"], (error, stdout, stderr) => {
  if (error) {
    console.error(`execFile error: ${error.message}`);
    return;
  }
  if (stderr) {
    console.error(`stderr: ${stderr}`);
    return;
  }
  console.log(`stdout: ${stdout}`);
});

This example demonstrates running the Node.js executable to get its version. execFile() is preferred when the command does not require shell features.

The `spawn()` method

Unlike the exec methods, which are primarily used for executing shell commands or executables and capturing their output, the spawn() method offers more control over child processes through stream-based output handling, direct interaction with executables, event-driven communication.

The spawn() method can be used in complex scenarios where direct interaction is necessary. spawn() launches a new process with a given command and provides streams i.e stdout, stderr for directly handling the process's output and errors.

Below is an example of how to use the spawn() method



const { spawn } = require("child_process");

const ls = spawn("ls", ["-lh", "/usr"]);

ls.stdout.on("data", (data) => {
  console.log(`stdout: ${data}`);
});

ls.stderr.on("data", (data) => {
  console.error(`stderr: ${data}`);
});

ls.on("close", (code) => {
  console.log(`child process exited with code ${code}`);
});

In this example, spawn() runs the ls command with specific arguments and attaches event listeners to handle the process’s output and exit status.

Running the code, you should see something like this:

The `fork()` method

The fork() method can be said to be a variation of spawn() designed for creating new Node.js processes. Unlike spawn(), which can launch any type of process, the fork() method is optimized for creating child processes that are themselves Node.js applications.

It provides an additional communication channel into the child process, allowing for simple message transmission between the parent and child processes.

Below is an example of how to use the fork() method.

The parent.js file:



const { fork } = require("child_process");

const child = fork("child.js");

child.on("message", (message) => {
  console.log(`Message from child: ${message}`);
});

child.send("Hello, child process!");

The child.js file:



process.on("message", (message) => {
  console.log(`Message from parent: ${message}`);
  process.send("Hello from child process!");
});

Here, the fork() creates a new Node.js process running the child.js script. It allows message passing between the parent and child process using send() and message events.

When you run the code, you should see something like this:

Differences between `fork()` and `spawn()`

Some of the differences between fork and spawn include:

Built-in communication channel: The fork() automatically sets up an IPC (inter-process communication) channel between the parent and child processes. In contrast, spawn() does not establish this channel by default; developers must manually configure IPC if needed.
Isolation and independence: Each child process generated by fork() is a separate Node.js process with its own V8 instance. The spawn() method, on the other hand, can launch any process, including Node.js apps, and does not provide the same level of isolation.
Use case specificity: The fork() technique is specifically intended for situations in which you need to generate worker processes that are part of the same application but operate concurrently. In contrast, spawn() is more general-purpose and is used when you need to launch arbitrary commands or scripts outside of the Node.js ecosystem.

Handling child process output

When a child process is created, it produces output in two main streams:stdout and stderr. These streams hold the results of the command or script run by the child process. To collect this output, add event listeners to the child process object for the data event on stdout and stderr.

Node.js provides you with several event listeners that are particularly useful for handling child process output and errors. Some of these include:

Data: Data event is emitted by the streams that represent the child process's stdout and stderr.
Error:The error event is triggered when an error occurs during the execution of the child process. This could be due to issues with the command itself or problems reading from stdout or stderr.
Close:The close event is emitted when the child process ends, indicating whether it terminated successfully or was killed.

Let's take a look at an example that shows how to spawn a child process, capture its output, and handle any errors that may occur.

We'll use the spawn() method to launch a simple command (echo) and listen for output and errors.



const { spawn } = require("child_process");

const child = spawn("echo", ["Hello, world"]);

child.stdout.on("data", (data) => {
  console.log(`stdout: ${data}`);
});

child.stderr.on("data", (data) => {
  console.error(`stderr: ${data}`);
});

child.on("error", (error) => {
  console.error(`Error occurred: ${error.message}`);
});

child.on("close", (code) => {
  console.log(`Child process exited with code ${code}`);
});

In this example, the parent process spawns a child process to execute the echo command, which simply prints Hello, world! to the terminal.

The parent process captures this output through the data event listeners attached to stdout and stderr. Additionally, it listens for the error event to catch any errors that might occur during the execution of the child process.

Finally, the close event indicates when the child process has finished executing, along with the exit code, which can be used to determine if the process was completed successfully.

Communication between processes

In distributed systems and concurrent programming, process communication is critical for component coordination. Node.js improves this communication with its child process module. This allows you to create child processes and facilitate inter-process communication (IPC).

This IPC mechanism, characterized by the exchange of data between parent and child processes, is pivotal for constructing modular and scalable applications.

There are several methods for facilitating; one that stands out is message passing via the send() method and message. This method leverages the send() function available on child process objects to send messages to child processes, and child processes listen for these messages using the message event.

Let's illustrate this with a practical example. We'll create a simple application where a parent process forks a child process and communicates with it using the send() method and message event.

The parent file, which is the parent.js, will look like this:



const { fork } = require("child_process");

const child = fork("./child.js");

child.send({ greeting: "Hello from parent!" });

child.on("message", (msg) => {
  console.log("Message from child:", msg);
});

console.log("Waiting for response...");

The child process file, which is child.js:



process.on("message", (msg) => {
  console.log("Message received from parent:", msg);

  process.send({ response: "Hello from child!" });
});

In this example, the parent process sends a message containing a greeting to the child process. The child process listens for this message, logs it, and then sends a response back to the parent process. The parent process also listens for incoming messages from the child process and logs them upon receipt.

This pattern of using send() and message events for IPC is particularly useful in scenarios where you need to coordinate actions between parent and child processes, such as sharing data, triggering actions, or implementing request-response patterns in a Node.js application.

Best practices for managing child processes

Below are some of the best practices used to manage the child process effectively:

Monitor system resources: one of the best practices is to regularly monitor the system's CPU, memory, and network usage to identify potential bottlenecks or resource leaks. Utilizing tools like top and ps can provide you with real-time insights into resource consumption.
Set timeouts for long-running processes: Use the timeout option in spawn() to automatically terminate child processes after a specified duration. This is beneficial for avoiding resource hogging by long-running activities.
Implement graceful shutdown procedures: Graceful shutdown procedures is when all requests to the server have been responded to, and no data processing work remains. Always ensure that child processes can be shut down gracefully, releasing resources and performing any necessary cleanup. This can be achieved by listening to termination signals and responding accordingly.
Apply the kill method: Use appropriate signals (e.g., SIGTERM for graceful shutdown, SIGKILL as a last resort). Verify that the process has been terminated successfully. Handle potential errors when applying the kill method.

Child process vs workers

While child processes are useful for some jobs, Node.js also provides another method for parallel execution. This method is known as workers, which is entirely different from the child process.

The worker threads module was introduced in Node.js v10.50. It provides a more efficient way to handle multithreading compared to child processes. Below is a table highlighting their differences

Feature	Worker threads	Child processes
Memory Isolation	Shares memory space with the parent process	Each child process has its own memory heap (complete isolation)
Communication	Uses message passing with less overhead (via SharedArrayBuffer)	Communicates via IPC channels with more overhead
Performance	More lightweight, lower overhead for frequent data exchange	Higher overhead, especially for frequent communication
Use Cases	Parallel computations, tasks requiring frequent data exchange	CPU-intensive tasks, running external scripts, isolated tasks
Error Isolation	Errors can affect the main thread	Errors in child processes do not affect the parent process
API Module	`worker_threads`	`child_process`
Creation Method	`new Worker()`	`spawn()`, `fork()`, `exec()`, `execFile()`
Multithreading	True multithreading with shared memory	Separate processes, no true multithreading
Overhead	Lower due to shared memory	Higher due to complete process creation
Resource Management	Managed within the same process	Separate resource allocation for each process

When to use the child process

Thechild process can be used when you need complete isolation, such as running external scripts or managing CPU-intensive tasks that could potentially crash the parent process.

When to use workers

The worker threads, on the other hand, can be used for tasks that benefit from shared memory and require frequent communication, such as data processing and parallel computations.

Conclusion

The child process module helps enable the execution of parallel tasks, running external scripts, and managing subprocesses efficiently. When you understand these methods and how to implement them, you will be able to significantly enhance your application's performance and scalability.

Six Tailwind CSS Utility Classes to Enhance Your Productivity

Oyedele Temitope — Mon, 26 Aug 2024 17:27:36 +0000

Tailwind CSS is one of the popular CSS frameworks that offers many classes. This classes helps to streamline and enhance web development workflows. Among the vast array of classes are some that developers have probably yet to hear of, underestimated, or are relatively new.

These classes possess immense potential to streamline development workflows, enhance the aesthetics of web interfaces, and boost productivity.

In this tutorial, we will examine six of these classes: the container class, size utility, space utility, line-clamp utility, ring utility, and truncate utility. We'll use Tailwind's CDN for this tutorial.

Container class

The container class allows you to create a container that scales its size based on your browser. It is designed to set an element's max-width to match the min-width of the current breakpoint in order to make it responsive to different screen sizes.

This responsiveness is achieved by adjusting the container's width based on the viewport size, which ensures that the content within the container is displayed appropriately across various devices.

To elaborate, Tailwind CSS uses a set of predefined breakpoints, such as sm, MD, lg, xl, 2xl, that correspond to specific minimum widths. These breakpoints apply different styles to different screen sizes and make it easier to create a responsive design without having to write custom media queries.

The container class uses these breakpoints to adjust its max-width accordingly to ensure that the content within the container scales and adapts to the browser's viewport size.

With this, your content is responsive and looks good on all devices without you needing to write custom CSS for each breakpoint.

Below is an example that demonstrates the container class:

<div class="container mx-auto px-4 border-2 border-gray-300 rounded-lg">
  <h1 class="text-4xl font-bold mb-4">Container Class</h1>
  <p class="text-lg">
    This is a demonstration of the container class in Tailwind CSS. The
    container is centered and scales its size based on the viewport size.
  </p>
  <div class="mt-8">
    <button
      class="bg-blue-500 hover:bg-blue-700 text-white font-bold py-2 px-4 rounded"
    >
      Click Me
    </button>
  </div>
</div>

When you check the result in your browser, you should have something like this:

You’ll see that the container's width will automatically adjust based on the current breakpoint. Your content will now be displayed appropriately across various devices.

Size utility

The size utility allows you to control an element's width and height simultaneously. This feature is particularly useful for creating square elements or ensuring that elements have consistent dimensions across your project.

The size utility provides various options, such as fixed pixel sizes, like size-48 for a specific pixel size, and predefined sizes from your Tailwind setup, e.g., size-2, which applies a width and height based on the scale defined in your Tailwind configuration.

Here’s how you can use the size utility:

<div class="container mx-auto px-4 py-8">
  <div class="grid grid-cols-3 gap-1">
    <div class="size-48 bg-green-500 flex items-center justify-center">
      <p class="text-white text-2xl font-semibold">Size 48</p>
    </div>
    <div class="size-64 bg-blue-500 flex items-center justify-center">
      <p class="text-white text-2xl font-semibold">Size 64</p>
    </div>
    <div class="size-80 bg-red-500 flex items-center justify-center">
      <p class="text-white text-2xl font-semibold">Size 80</p>
    </div>
  </div>
</div>

For the first box, the size-48 sets both width and height to 48 of the spacing scale. The second and third boxes follow a similar structure, with size-64 and size-80classes intended to set their sizes.

When you check the result in your browser, you should have something like this:

Space utility

The space utility is designed to control the spacing between elements, making it easier to create visually appealing layouts with consistent spacing.

Tailwind provides two primary classes for managing space: space-x for horizontal spacing and space-y for vertical spacing. These classes can be applied to your container element to automatically apply spacing between its direct child elements.

This is crucial for maintaining consistent spacing throughout your design. It saves time by eliminating the need to write custom CSS for spacing, allowing you to focus on other aspects of your design.

Below is an example of how to use the space utility to add horizontal spacing between buttons within a flex container:

<div class="container mx-auto px-4 py-8">
  <div class="grid grid-cols-3 gap-4">
    <div
      class="p-6 max-w-sm mx-auto bg-white rounded-xl shadow-lg flex flex-col items-center space-y-4"
    >
      <div>
        <p class="text-xl text-black font-medium">Card 1 Title</p>
        <p class="text-base text-gray-500">
          Card 1 description or additional information.
        </p>
      </div>
    </div>
    <div
      class="p-6 max-w-sm mx-auto bg-white rounded-xl shadow-lg flex flex-col items-center space-y-4"
    >
      <div>
        <p class="text-xl text-black font-medium">Card 2 Title</p>
        <p class="text-base text-gray-500">
          Card 2 description or additional information.
        </p>
      </div>
    </div>
    <div
      class="p-6 max-w-sm mx-auto bg-white rounded-xl shadow-lg flex flex-col items-center space-y-4"
    >
      <div>
        <p class="text-xl text-black font-medium">Card 3 Title</p>
        <p class="text-base text-gray-500">
          Card 3 description or additional information.
        </p>
      </div>
    </div>
  </div>
</div>

In the code above, the space-y-4 utility applies vertical spacing between the child elements of each card, thereby creating consistent spacing elements inside each card.

When you check the result in your browser, you should have something like this:

Line-clamp utility

The line-clamp utility is a powerful tool for controlling text overflow. It helps by visually truncating text after a fixed number of lines. It is particularly useful when you want to maintain a clean and uniform layout, especially when you're dealing with dynamic content that might exceed the desired display area.

Below is an example of a card that uses the line-clamp utility to control text:

<div class="max-w-sm rounded overflow-hidden shadow-lg m-4">
  <img class="w-full" src="https://via.placeholder.com/150" alt="Card image" />
  <div class="px-6 py-4">
    <div class="font-bold text-xl mb-2">Card Title</div>
    <p class="text-gray-700 line-clamp-3">
      Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec nec dolor
      et velit aliquam efficitur. Sed velit nisi, lacinia eu nisl id, lacinia
      lacinia nisl.
    </p>
  </div>
  <div class="px-6 pt-4 pb-2">
    <span
      class="inline-block bg-gray-200 rounded-full px-3 py-1 text-sm font-semibold text-gray-700 mr-2 mb-2"
      >#tag1</span
    >
    <span
      class="inline-block bg-gray-200 rounded-full px-3 py-1 text-sm font-semibold text-gray-700 mr-2 mb-2"
      >#tag2</span
    >
  </div>
</div>

The description text is controlled using the line-clamp-3 class, which limits the text to three lines. If the text exceeds three lines, it will be truncated, and an ellipsis will be added to indicate the truncation.

This ensures that the card remains visually clean and that users can quickly understand the content without being overwhelmed by too much text.

When you check the result in your browser, you should have something like this:

Ring utility

The ring utility is used to apply a border around an element. It also provides a way to add outline shadows or focus rings to your elements. It's a nice alternative to the older shadow-outline and shadow-xs classes, allowing for more customizable focus states.

The ring utility enhances the user experience by providing visual feedback on interactive elements, such as buttons or input fields, without the need for custom CSS. It is highly customizable and it allows you to control the width, color, and opacity of the ring.

Below is an example of how you can use the ring utility:

<div class="bg-black min-h-screen flex items-center justify-center">
  <div class="flex flex-col items-center space-y-4">
    <button
      class="bg-blue-500 text-white px-4 py-2 rounded ring-2 ring-blue-300 border border-white hover:ring-blue-500 mr-2 focus:ring-4 focus:ring-blue-500"
    >
      Button 1
    </button>
    <button
      class="bg-green-500 text-white px-4 py-2 rounded ring-2 ring-green-300 border border-white hover:ring-green-500 mr-2 focus:ring-4 focus:ring-green-500"
    >
      Button 2
    </button>
    <button
      class="bg-red-500 text-white px-4 py-2 rounded ring-2 ring-red-300 border border-white hover:ring-red-500 mr-2 focus:ring-4 focus:ring-red-500"
    >
      Button 3
    </button>
  </div>
</div>

In the code above, the ring utility is used to apply a ring outline around the button elements, which can be customized in terms of width and color.

Not only that, it can be combined with other utilities to change the ring's appearance based on different states, such as hover or focus.

With this approach, you get interactive and accessible designs that provide visual feedback to users when they interact with the buttons.

When you check the result in your browser, you should have something like this:

Truncate utility

The truncate utility is one of Tailwind's text overflow utilities used to truncate text that overflows its container by hiding the extra content and replacing it with an ellipsis (...).

Its main purpose is to ensure that texts do not spill out of their designated area so you can maintain a clean and professional appearance. It also saves you time by preventing layout issues caused by overflowing text.

Below is an example showing how to use the truncate utility:

<div class="w-full max-w-lg bg-white shadow-lg rounded-lg p-6 mt-10">
  <h2 class="text-2xl font-bold mb-4">Card Title</h2>
  <p class="truncate">
    Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed euismod, nunc
    at cursus pellentesque, nisl eros pellentesque quam, a faucibus nisl nunc id
    nisl.
  </p>
</div>

The truncate class is applied to the <p> tag to truncate the text with an ellipsis if it overflows its container.

When you check the result in your browser, you should have something like this:

And that's a wrap!

Conclusion

In this article, we examined six utility classes that can boost productivity and provided an example for each.

Understanding these utility classes can help you focus more on creating unique and functional designs rather than spending excessive time on repetitive CSS coding tasks.

Top Golang Blogs for Learning and Building

Oyedele Temitope — Sun, 11 Aug 2024 23:28:27 +0000

As a newbie in Golang, searching for the right resources to learn and up your game can be frustrating. You’ll need articles to break down the complex concepts from the documentation or even tutorials to help build stuff. This problem isn't only faced by newbies but also by developers at the intermediate and senior levels.

In this article, I have curated a list of ten blogs that will help you learn and grow. The blogs listed below include articles on learning complex concepts and tutorials that help you build and familiarize yourself with Golang.

1. The Go Blog

The official Go blog is an invaluable resource for Golang developers. It is maintained by Google's Go team and contains detailed articles on language features, best practices, and upgrades. You will find in-depth explanations of new releases, performance improvements, and insights into the language's design philosophy.

2. Twilio Blog

While not solely dedicated to Go, Twilio's blog provides excellent Golang tutorials. If you're a developer looking to not only learn Golang but also build amazing stuff with it, then the Twilio blog is the right place. It also boasts resources for building communication-related applications. Their Go articles cover topics like building SMS and voice applications and demonstrating how to integrate Go with Twilio's powerful APIs.

3. Dev.to

Dev.to has a thriving community of developers who share their skills and experiences. The portal includes several Go articles, spanning beginner tutorials to advanced topics. Developers from a variety of backgrounds contribute practical guides, project walkthroughs, and debates on Go best practices.

4. A Journey With Go

This blog, which is authored by Vincent Blanchon, provides an in-depth look at the internals of Go. It explores advanced topics like garbage collection, concurrency patterns, and compiler optimizations. If you're looking to understand the nitty-gritty details of how Go works under the hood, this blog is an excellent resource.

5. Go by Example

Go by example provides a hands-on approach to learning Go. It offers a collection of annotated example programs covering a wide range of Go's features and standard libraries. Each example is concise and focused, making it an ideal resource for both beginners and developers looking for quick references.

6. Ardan Labs

The Ardan lab's blog is managed by Bill Kennedy and his staff. It is well-known for its high-quality Go content. Ardan Labs blog discusses advanced subjects like performance optimization, design patterns, and best practices for developing production-ready Go applications. Their articles often include detailed code examples and explanations.

7. Gopher Academy Blog

The Gopher Academy Blog is a community-driven platform that publishes articles from various Go specialists. It discusses a variety of topics, including web development, testing methodologies, and Go tools. The blog also includes information about Go conferences and community events.

8. Golang Weekly

Golang Weekly is a curated newsletter that delivers a weekly summary of the most important Go news, articles, tools, and projects. While not a typical blog, it is an outstanding repository of Go-related content from many sources. Subscribers receive a brief email every Thursday with the most recent tutorials, blog posts, package announcements, and job opportunities in the Go ecosystem. It is a fantastic resource for keeping up with the quickly evolving Go community and discovering fresh learning resources from varied sources.

9. Dave Cheney’s Blog

Dave Cheney is a prominent figure in the Go community. He likes to share his extensive knowledge through his personal blog. His articles cover advanced topics, performance tips, and insightful discussions on Go's design principles. The blog is especially useful for intermediate to advanced Go developers who want to enhance their understanding of the language.

10. Applied Go

Maintained by Christoph Berger, applied Go focuses on practical applications of Go programming. The blog contains a combination of tutorials, hints, and in-depth articles about various Go topics. It's especially useful for developers trying to bridge the gap between theory and practice, as the content ranges from fundamental concepts to advanced approaches. The blog frequently offers comprehensive, working examples that show how to use Go in real-world applications.

Conclusion

These ten blogs offer a wealth of knowledge for Go developers at all levels. From official documentation and community-driven content to specialized topics and practical projects, these resources will help you enhance your Go programming skills and stay current with the latest developments in the Go ecosystem.

It's also important to note that these blogs aren't the only ones available to learn and master Golang. There are tons of others. If you think I missed a blog, you can add yours in the comments section. Do not forget to share if you found this helpful.

Three Ways to Cause Infinite Loops When Using UseEffect in React and How to Prevent Them

Oyedele Temitope — Sun, 21 Jul 2024 15:56:50 +0000

The useEffect hook is a powerful tool for performing side effects and handling cleanup in your functional components. However, if used incorrectly, it can lead to infinite loops that can crash your application.

In this article, we will explore three common ways infinite loops can occur when using useEffect and provide guidance on preventing them.

Forgetting to Add the Dependency Array in UseEffect

One of the most common mistakes that can lead to an infinite loop is forgetting to provide a dependency array as the second argument to the useEffect hook.

The dependency array is crucial because it tells React to only re-run the effect when one of the dependencies has changed. If you omit the dependency array, React will default to running the effect on every render, which can cause an infinite loop.

Below is an example of how this can happen:

export default function Counter() {
  const [count, setCount] = useState(0);

  useEffect(() => {
    setCount(count + 1);
  });

  return <div classname="App">value:{count}</div>;
}

In the above code, we have a Counter() component that uses the useEffect hook to update the count state every time the component renders.

During the initial render, the useEffect hook will run as expected. However, if the state is updated inside the useEffect without adding the dependency array, the component will re-render. Since there's no dependency check, the useEffect hook will also run again after the re-render.

This cycle of state update triggering re-render, which in turn triggers useEffect again, will continue indefinitely, causing the infinite loop.

If you check the result in the console of your browser, You can see you get a warning maximum data exceeded, and it’s going on continuously:

How to Prevent It

To prevent this infinite loop, you need to provide a dependency array that includes the values on which the effect depends. In this case, the effect does not depend on any changing values, so the dependency array should be empty:

export default function Counter() {
  const [count, setCount] = useState(0);

  useEffect(() => {
    const interval = setInterval(() => {
      setCount((prevCount) => prevCount + 1);
    }, 1000);

    return () => clearInterval(interval); /
  }, []); 

  return <div className="App">value: {count}</div>;
}

By doing this, React will only re-run the effect when the count state changes, preventing the infinite loop.

Using an Object as a Dependency

Another way to accidentally create an infinite loop is by passing an object as a dependency in the useEffect hook. In JavaScript, objects are passed by reference, which means that even if the object's values don't change, the reference to the object itself is considered a change. This can lead to the effect running on every render, causing an infinite loop.

Below is an example of how this can happen:

export default function Counter() {
  const [count, setCount] = useState(0);
  const object = { name: "tope", age: 23 };

  useEffect(() => {
    setCount((count) => count + 1);
  }, [object]);

  return <div className="App">value of count:{count}</div>;
}

In this code above, the object variable is used as a dependency in the useEffect hook. Since objects in JavaScript are reference types, the useEffect hook will run on every render because the reference to the object changes on every render, even if the content of the object remains the same. This will cause the count state to update on every render, leading to an infinite loop.

How to Prevent It

To prevent the infinite loop caused by using an object as a dependency in the useEffect hook, you can use the useMemo hook to memoize the object.

It ensures that the object's reference does not change on every render unless its actual data changes:

const object = useMemo(() => ({ name: "tope", age: 23 }), []); // Memoize the object

useEffect(() => {
  setCount((prevCount) => prevCount + 1);
}, [object]);

⚠️ Don't forget to import the useMemo hook.

The useMemo hook memoizes the object, ensuring that its reference does not change on every render. It prevents the useEffect hook from running on every render, thus avoiding the infinite loop.

Passing a Function as a Dependency

Passing a function as a dependency to useEffect can also lead to infinite loops. This is because functions in JavaScript are also passed by reference, so even if the function's body doesn't change, the reference to the function itself will be considered a change on every render.

Below is an example of how this can happen:

export default function Counter() {
  const [count, setCount] = useState(0);
  const handleClick = () => {};

  useEffect(() => {
    setCount((count) => count + 1);
  }, [handleClick]); // Passing a function as a dependency

  return (
    <div className="App">
      <p>value of count: {count}</p>
    </div>
  );
}

In the code above, the handleClick function is passed as a dependency to the useEffect hook. Since functions in JavaScript are reference types, the reference to handleClick changes on every render, which causes the useEffect hook to run on every render and leads to an infinite loop.

How to Prevent It

To prevent this infinite loop, you should avoid passing functions directly as dependencies to useEffect. Instead, you can use the useCallback hook from React to create a memoized version of the function that only updates when its dependencies change:

const handleClick = useCallback(() => {
  // Function logic here
}, []); // Dependencies of the function

useEffect(() => {
  setCount((prevCount) => prevCount + 1);
}, [handleClick]);

⚠️ Don't forget to import the useCallback hook.

The useCallback, will ensure that handleClick will only update when its dependencies change, preventing the infinite loop.

Conclusion

In this article, we explored three ways to avoid infinite loops when using the useEffect hook in React. Infinite loops can be a common issue while working with useEffect. Therefore, it's important to always use practices that can help prevent infinite loops and improve the performance of your React applications.

DEV Community: Oyedele Temitope

Why “Ship First” Fails in AI and How High-Performing Teams Build Instead

Why Traditional Engineering Logic Breaks in AI Systems

You Don’t Need More Prompts. You Need a System

Start With the User, Not the Model

Defining the Test Space for AI Systems

Coverage Metrics

Failure Mode Categories

Guardrail Rules

Business-Critical Paths

Evaluation Metrics

Building and Validating the System After Writing Tests

Pipeline Construction

Real Data Validation

Closing the Gaps the Tests Expose

Best Practices Teams Use to Keep Their Systems Dependable

Wrapping Up

15 Go Packages Worth Your Time

Core Utilities

1. spf13/cobra

2. urfave/cli

3. stretchr/testify

4. mitchellh/mapstructure

5. olekukonko/tablewriter

Web & APIs

6. gin-gonic/gin

7. labstack/echo

8. go-chi/chi

9. tidwall/gjson

Dev & Ops Helpers

10. golangci/golangci-lint

11. sirupsen/logrus

12. uber-go/zap

Storage, Persistence, and Data

13. jmoiron/sqlx

14. etcd-io/bbolt

15. go-yaml/yaml

Concusion

How to Scrape Products from a Page with Infinite Scroll via 'Load More' Button

Prerequisites

Project Scope

Access the Content

Load More Products

Parse Product Information

Export Product Information to CSV

That’s How To Scrape Data in Node.js using Puppeteer and Cheerio

How to create an AI Avatar in Go using Simli

Prerequisites

Set up your Environment

Build the Project

Import Required Packages and Set Data Structure for Response

Generate the AI Avatar

Add the Main Function

Create the Static File

Test the application

That's how to create an AI avatar in Go using Simli

Forward Proxy vs Reverse Proxy vs Load Balancers

What Is a Forward Proxy?

Why Forward Proxies Are So Important

What Is a Reverse Proxy?

Why Use a Reverse Proxy?

What Is a Load Balancer?

Why Use a Load Balancer?

Difference Between Forward Proxies, Backward Proxies, and Load Balancers

Conclusion

How to Set Up Authorization in a Bookstore Management System with Go, HTMX, and Permit.io

Table of Content

Prerequisites

Project Scope

Project Setup

Setup the Database

Install Dependencies

Setup your PDP (Policy Decision Point) Container

Build the Application

Configure Database Connection

Create the Handlers

Create Authorization Middleware

Create the Models

Create the HTML Templates

Test the Application

1. `spf13/cobra`

2. `urfave/cli`

3. `stretchr/testify`

4. `mitchellh/mapstructure`

5. `olekukonko/tablewriter`

6. `gin-gonic/gin`

7. `labstack/echo`

8. `go-chi/chi`

9. `tidwall/gjson`

10. `golangci/golangci-lint`

11. `sirupsen/logrus`

12. `uber-go/zap`

13. `jmoiron/sqlx`

14. `etcd-io/bbolt`

15. `go-yaml/yaml`

What is React `cache` Function

Benefits of the `cache` Function

What is React `cache` Function

Benefit of the `cache` Function

The `exec()` method

The `execFile()` method

The `spawn()` method

The `fork()` method

Differences between `fork()` and `spawn()`