DEV Community: Alex Nguyen

Orama search with Astro

Alex Nguyen — Wed, 27 Dec 2023 06:20:23 +0000

Orama is a search engine that you can integrate with most, if not any, JavaScript site framework. In this tutorial, we'll cover how to add Orama to an Astro site.

Not using Astro? The Orama SDK is framework-agnostic, so you should be able to apply the following methods with other frameworks.

Static search from the build folder

Setup

Orama has an Astro integration for indexing the content for any static Astro build:

npm install @orama/plugin-astro

In your Astro config, add the orama integration:

import orama from "@orama/plugin-astro";

export default defineConfig({
  integrations: [
    orama({ ... }),
  ],
})

Orama requires a basic configuration object:

pathMatcher is how you can select the static routes you want to index with a regular expression
language is the output language
contentSelector sets which elements to target - by default, <body> is used which will include unrelated content like the header and footer, which may not be what you want

import orama from "@orama/plugin-astro";

export default defineConfig({
  integrations: [
    orama({
      search: {
        pathMatcher: /^blog\/.+$/,
        language: "english", // optional - default is English 
        contentSelectors: ["h1", "article"], // default is body
      },
    })
  ],
})

In this example, I'm also only targeting /blog/* pages, and not including /blog/.

For simplicity, we'll create a separate search.astro page to demonstrate some basic search functionality. Add a search input and a placeholder element for any search results:

---
---
<input type="text" id="search" name="search" />

<div id="search-results"></div>

We'll load the database, run Orama's search function against it every time the search input changes, and display any results:

<script>
  import { getOramaDB, search } from "@orama/plugin-astro/client";
  const db = await getOramaDB("search");

  const searchInput = document.querySelector<HTMLInputElement>("input#search");

  searchInput?.addEventListener("input", async (e) => {
    const searchResults = document.querySelector("#search-results");
    const results = await search(db, { term: e.target.value });

    searchResults.innerHTML = ""; // reset displayed results

    for (const result of results.hits) {
      searchResults.innerHTML += `
      <a href="${result.document.path}">
        ${result.document.title}
      </a>`;
    }
  });
</script>

For an example of this search in action, check out my search page - code is available here.

Testing in development

By default, this won't work during development as Orama has to read the database file generated during the build process. This is a common problem with static search solutions where you can't easily test things in development.

For a workaround, build your site, and copy your database file (oramaDB_[dbname].json) from your dist/assets/ folder into an assets folder in your public folder (/public/assets/). This will allow you to work with your data (or some test data at least) in development.

After you've finished working on your search page or component, you can just delete this file or folder, or leave it. You could also add the file to your .gitignore so it doesn't get committed.

Limitations

Using the Orama plugin has a few limitations. If you use the page title for the search result headings, you will likely want to extract out the real title without the site title or any dashes/separators.

It's also not very flexible - for example, if you want to add excerpts or other custom data.

Building a search database

We can use a server endpoint to generate a custom database ourselves with a little more work. This database could have data from your Astro collections or from any other source (like an API). This endpoint will be called by our page's script.

Server

Start by installing the Orama TypeScript SDK:

npm install @orama/orama

Create an endpoint file called [term].ts in /pages/api/search (create the api and search folders) and add:

export const prerender = false; // for output: "hybrid" sites
import { getCollection } from "astro:content";

export async function GET() {
  const posts = await getCollection("posts");
}

Like with the previous example, we'll search against blog posts. We'll query the posts collection and return the data as-is. Then we'll create an Orama database, and insert our collection data into it:

export const GET: APIRoute = async ({ params }) => {
  const posts: never[] = await getCollection("posts");

  const db = await create({
    schema: {
      id: "string",
      slug: "string",
      body: "string",
      collection: "string",
      data: {
        title: "string",
        tags: {
          slug: "string",
          collection: "string",
        },
        description: "string",
        published: "boolean",
      },
    },
  });

  await insertMultiple(db, posts);

  const results = await search(db, {
    term: params.term ?? "",
  });

  return new Response(JSON.stringify(results));
};

The schema defined here is based on the posts collection - adjust it according to your own data. Anything not defined in the schema won't be included in the database, so you can actually refine/reduce the data at a schema level instead of modifying the data directly.

Go to http://localhost:4321/api/search?term= and enter some terms to test things out.

To display the results on a page, we can use the previous example, and change things around a bit:

<script>
  const searchInput = document.querySelector<HTMLInputElement>("input#search");

  searchInput?.addEventListener("input", async (e) => {
    const searchResults = document.querySelector("#search-results");
    const response = await fetch(`/api/search?term=${e.target.value}`);
    const results = await response.json();

    searchResults.innerHTML = ""; // reset displayed results

    for (const result of results.hits) {
      searchResults.innerHTML += `
      <a href="/blog/${result.document.slug}">
        ${result.document.data.title}
      </a>`;
    }
  });
</script>

Head to http://localhost:4321/search to make sure everything works similar to the Orama integration example.

Static

We can also generate a database statically. We can do this by fetching or getting the data in an Astro page or component - the database will be generated server-side during development mode (meaning you can test things without needing any workarounds), and statically at build-time.

In an Astro page or component, fetch the data you want (again, this could be from a collection or from another source like an API):

---
import { getCollection } from "astro:content";
const posts = await getCollection("posts");
---

Next, we'll pass this data to our script so we can use it with Orama - it'll need to be serialized from server to client:

<input type="text" id="search" name="search" />

<div id="search-results"></div>

<astro-search data-posts={JSON.stringify(posts)}></astro-search>

<script>
  class AstroSearch extends HTMLElement {
    constructor() {
      super();

      const posts = JSON.parse(this.dataset.posts);
    }
  }

  customElements.define("astro-search", AstroSearch);
</script>

Finally, we'll add the search functionality (similar to the previous scripts):

<script>
  import { create, insertMultiple, search } from "@orama/orama";

  const db = await create({
    schema: {
      id: "string",
      slug: "string",
      body: "string",
      collection: "string",
      data: {
        title: "string",
        tags: {
          slug: "string",
          collection: "string",
        },
        description: "string",
        published: "boolean",
      },
    },
  });

  class AstroSearch extends HTMLElement {
    constructor() {
      super();

      const posts = JSON.parse(this.dataset.posts);

      insertMultiple(db, posts);

      const searchInput =
        document.querySelector<HTMLInputElement>("input#search");

      searchInput?.addEventListener("input", async (e) => {
        const searchResults = document.querySelector("#search-results");
        const results = await search(db, { term: e.target.value });

        searchResults.innerHTML = ""; // reset displayed results

        for (const result of results.hits) {
          searchResults.innerHTML += `
          <a href="/blog/${result.document.slug}">
            ${result.document.data.title}
          </a>`;
        }
      });
    }
  }

  customElements.define("astro-search", AstroSearch);
</script>

Preloading images with Astro

Alex Nguyen — Thu, 14 Dec 2023 19:58:37 +0000

It is fairly straightforward to preload images in an Astro site that are served from your public folder - you just need to get the path to the image, and add it to a link rel="preload" tag:

<head>
  <link rel="preload" href="/image.png" as="image" /> <!-- /public/image.png -->
</head>

But how do you preload images that are loaded with the Image or Picture components if you need them to be optimized? There's no simple way to do this, like with Next.js's Image priority feature.

But you can use Astro's getImage function to generate an optimized image server-side (or during the build for static sites), and use this image's path in the preload's href and on the page (as an <img> or picture>) instead.

Example

In this example, we'll preload an image from our /src/assets folder called project.png.

Start by importing the getImage function and the image in an Astro page or component:

---
import { getImage } from "astro:assets";

import image from "../assets/project.png".
const optimizedImage = await getImage({
  src: image,
  format: "webp",
  width: 500, // resize optional 
  height: 500, // resize optional
});
---

To use this optimized image, you can still use the <Image /> component:

<Image
  src={optimizedImage.options.src}
  width={optimizedImage.attributes.width}
  height={optimizedImage.attributes.height}
  alt=""
/>

You can also use a normal <img> tag:

<img
  src={optimizedImage.src}
  width={optimizedImage.attributes.width}
  height={optimizedImage.attributes.height}
  alt=""
/>

Finally, add a preload with the image to your page's head element:

<head>
  <link rel="preload" href={optimizedImage.src} as="image" />
</head>

If you're using a layout with an existing <head>, then you can use named slots to pass in the link tag from another page or component without creating duplicate <head> tags:

<head>
  {/* ... */}

  <slot name="head" />
</head>

<Fragment slot="head">
  <link rel="preload" href={optimizedImage.src} as="image" />
</Fragment>

Content Collections

If you're using content collections, and you're including images in the collection schema, then you can reference this image directly:

---
import { getImage } from "astro:assets";

/* export async function getStaticPaths() { ... }, etc. */

const { project } = Astro.props;
const optimizedImage = await getImage({ src: project.data.image });
---

Bringing it all together

To check that the preloading worked correctly, you can build your site and run it locally - change the commands as needed based on your package manager.

For static sites, run npm run build and then npm run preview.

For hybrid or server sites, you should be able to use the Node.js adapter (even if you're using another adapter like for Vercel or Netlify, just swap them out for testing). Run npm run build and then npm run preview.

There doesn't seem to be a simple built-in way of checking preloaded resources. But if you open your browser's network tab and filter for images, the preloaded image should be at or near the top of all requests:

You can also remove the filter to see how it compares with all other request types, or re-build with the preload removed to see how it changes.

Make sure to also check the page source - the paths in the <link> tag and <img> tag should match:

<link rel="preload" href="/_astro/project.xCZHVmaS_29nlmI.webp" as="image">

<img src="/_astro/project.xCZHVmaS_29nlmI.webp" />

Also, if you're using Chrome or Edge, you'll get a warning in your console after a few seconds if it didn't work - The resource *.webp was preloaded using link preload but not used within a few seconds .... Other browsers may do something similar.

View Transitions

If you're using View Transitions and animations between images, preloading should also help prevent the image flickering as it's loaded for the first time as you navigate to the new page.

For example, if you have a product carousel with small images linking to product pages with a larger resolution of the same image, you may want to preload these larger images.

Otherwise, it can look jarring when the image loads during the element's transition to the next page. Preloading the image should help with this. Alternatively, you could use the same image between pages if you're willing to sacrifice flexibility with the optimizations e.g. width and height.

Keep in mind that preloading isn't a guarantee for improving things, and may actually make your page performance worse. But the example in this post is a reasonable solution if you feel the need to use preloading or other optimization techniques.

Feel free to follow me or check out my blog.

Custom client directives with Astro

Alex Nguyen — Wed, 06 Dec 2023 23:41:02 +0000

Custom client directives are useful when you want more control over when framework components or islands hydrate in an Astro site. A direct benefit is potentially improved page performance by only loading scripts when they're actually needed.

Use case

It's common to have two navigation menus - one for mobile and one for desktop. If the mobile navigation is toggleable, client:visible can be used, so it hydrates at a low priority.

However, if the desktop navigation has dropdown menus that don't need to hydrate at all even when visible, then that component could use hover or mouseover based client directive instead, so it only hydrates based on a user action.

For example, I used this at vsctrust.org.nz. Using client:visible in a local production build:

Using client:mouseover:

Did it improve the performance to a level where I could perceive it? No, but it was a fun exercise, and maybe doing something like this would make a real difference on a heavier page, especially on older devices.

Creating a custom client directive

We'll look at creating a custom hover or mouseover client directive.

Somewhere in your Astro project, (like a lib folder or similar), create a folder called client-directives. In this folder, create a mouseover.js (or mouseover.ts if you prefer TypeScript) file and add:

/**
 * @type {import('astro').ClientDirective}
 */
export default (load, opts, element) => {
  element.addEventListener(
    "mouseover",
    async () => {
      const hydrate = await load();
      await hydrate();
    },
    { once: true }, // remove the event listener 
  );
};

import type { ClientDirective } from "astro";

export default (function (load, _opts, element) {
  element.addEventListener(
    "mouseover",
    async () => {
      const hydrate = await load();
      await hydrate();
    },
  { once: true },
);
} satisfies ClientDirective);

This will hydrate the component once the mouseover event is triggered on the element.

Then add this directive as an Astro integration. Create a register.js or register.ts file and add:

/**
 * @type {() => import('astro').AstroIntegration}
 */
export default () => ({
  name: "client:mouseover",
  hooks: {
    "astro:config:setup": ({ addClientDirective }) => {
      addClientDirective({
        name: "mouseover",
        entrypoint: "./src/lib/client-directives/mouseover.js",
      });
    },
  },
});

import type { AstroIntegration } from "astro";

export default (): AstroIntegration => ({
  name: "client:mouseover",
  hooks: {
    "astro:config:setup": ({ addClientDirective }) => {
      addClientDirective({
        name: "mouseover",
        entrypoint: "./src/lib/client-directives/mouseover.ts",
      });
    },
  },
});

We'll also add types for the directive. Create an index.d.ts file and add:

import "astro";
declare module "astro" {
  interface AstroClientDirectives {
    "client:mouseover"?: boolean;
  }
}

For WebStorm at least, this doesn't seem to affect any of the types or warnings that are picked up once you use a custom client directive, but it doesn't hurt to add this anyway.

Bringing it all together

Finally, we'll add the client directive to our Astro config:

import mouseoverDirective from "/src/lib/client-directives/register";

export default defineConfig({
  integrations: [
    mouseoverDirective(),
    // ...
  ],
});

We can add it to any component like so:

<ReactComponent client:mouseover />
{/* You may get warnings about using it in your IDE (for example, that it requires a value in WebStorm), you could explicitly pass it a value of `true` (which doesn't affect anything but removes the warning) or just leave it. */}

import { useEffect } from "react";

export function ReactComponent() {
  useEffect(() => {
    const content = document.querySelector("#content");
    if (content) content.textContent = "Hydrated";
  }, []);

  return <div id="content">Hover over me</div>;
}

Then restart your development server for the client directive to be loaded.

To see if it's working, open up your browser's network tab, and check if this component's scripts are fetched once you hover over it.

You may not ever need to use a custom client directive, but it's a pretty cool feature that highlights Astro's flexibility.

Feel free to follow me or check out my blog.

All about Astro

Alex Nguyen — Wed, 06 Dec 2023 23:31:40 +0000

Astro has recently become my favourite framework for building websites. It has a great developer experience, features, community, and it's frequently updated.

These are some findings or quirks I've noted when using it that don't warrant a separate blog post, things that you may encounter depending on the Astro version you're using. These may be things I've missed in the documentation, something I've done incorrectly (likely), or an actual issue or bug (less likely).

Upgrading to Astro 4.0

astro-icon with local images

Update to astro-icon@0.8.2 to be able to use local icons in an Astro 4.0 project.

View Transitions

client:only priority

client:only is meant to have a high priority and behaves like client:load (except server rendering is skipped), but any component using client:only won't immediately show its static content (without hydration) resulting in a flash (unlike client:load). This doesn't seem to change if your page is server rendered or static.

Selectors with page events

When selecting elements, make sure to call them inside the called function of the event listener, otherwise the reference will be stale when you get to the next page:

<script is:inline>
  // this won't work after you navigate
  // const menu = document.getElementById("menu");

  function initMenu() {
    const menu = document.getElementById("menu");
    // ...
  }

  initMenu();

  document.addEventListener("astro:after-swap", () => initMenu());
</script>

define:vars with lifecycle events

If you use define:vars to pass frontmatter variables into your script tag, everything inside will re-run everytime that variable changes:

---
const pathname = new URL(Astro.request.url).pathname;
---

<script define:vars={{ pathname }}>
  // this will log for every pathname change, not just once on initial site load
  document.addEventListener("astro:page-load", () => console.log("page-load"), {
    once: true,
  });
</script>

Issues with CSS animations

There is a visual issue I've encountered with Firefox when using <ViewTransitions /> with the default fallback (for browsers that don't support the View Transitions API yet) and a CSS opacity animation on page load.

The opacity was set to 0 and set to animate to 100% with a CSS animation but using View Transitions resulted in the animation never running (or the opacity not being set).

The workaround was to use a fallback of swap:

<head>
  <ViewTransitions fallback="swap" />
  <!-- ... --> 
</head>

This is something I'll look to investigate in the future.

Collections

Querying all collections

There is no built-in way to query all collections. One workaround to do this is using Astro.glob to pull all markdown files:

---
const collections = await Astro.glob("../content/**/*.md");
---

The data returned from this will be different from the data returned from getCollection. If you needed it in the same structure, you could extract the folder names only using the data from Astro.glob, and then loop over them with getCollection.

Images

Remote images with subdomains

You need to specify the subdomain for any remote image links in astro.config.mjs, otherwise it won't get optimized and the original source will be used:

<!-- resolves to https://fastly.picsum.photos/** -->
<Image
  src="https://picsum.photos/200/300"
  width="500"
  height="500"
  alt="remote image"
/>

export default defineConfig({
  image: {
    domains: ["fastly.picsum.photos"],
    // or if you need a wildcard
    remotePatterns: [{
      protocol: 'https',
      hostname: '**.picsum.photos'
    }]
  }
});

Content collections

When referencing images in content collections using the image helper, you can also reference the image from the root directory (src).

---
image: ./my-post.jpg # default
image: /src/content/posts/my-post.jpg
---

You may need to do this for compatibility with your CMS, for example, if it doesn't support reading relative paths for things like image previews.

Framework Components / Islands

Mixing frameworks

When using components from multiple frameworks together, not adding the extension may result in an error of This component likely uses @astrojs/react ...:

import { Gallery } from "./Gallery";

Change this to:

import { Gallery } from "./Gallery"; // React
import { Gallery } from "./Gallery.vue"; // Vue
import { Gallery } from "./Gallery.svelte"; // Svelte

Alternatively (although I'm not sure why you would do this), you can force the component to hydrate for a particular framework with client:only to make it work without the extension.

Svelte without client directive

You must use a client directive for a Svelte component's slot to appear, even if this component doesn't need to hydrate.

<Button client:load>Button Text</Button>

<button>
  <slot />
</button>

The same behaviour also happens with named slots.

<Button>
  <Fragment slot="text">Button Text</Fragment>
</Button>

<button>
  <slot name="text" />
</button>

You don't need to do this with React or Vue.

TypeScript

Props with client directives

You may get various type errors when using client directives with any props other than children (not sure if this applies to other non-React frameworks):

<Fade client:visible delay={0.25}><h1>Title</h1></Fade>

Which results in:

Type '{ children: any; "client:visible": true; }' is not assignable to type 'IntrinsicAttributes & { delay: number; children: ReactNode; }.
Property 'delay' is missing in type '{ children: any; "client:visible": true; }' but required in type '{ delay: number; children: ReactNode; }'

A (bad) workaround is to mark these prop(s) as optional in your type or interface inside that component:

export function Fade({
  delay,
  children,
}: {
  delay?: number;
  children: React.ReactNode;
}) {}

There is likely a better solution as this hides the error even if the prop(s) isn't optional.

WebStorm

There are some issues resolving types with WebStorm with an active issue here.

One solution which solves a lot of the problems is using the bundled TypeScript setting. Open Settings > Preferences > Languages & Frameworks > TypeScript > Bundled.

Miscellaneous

Importing the Astro config

If you want to import your config from astro.config.mjs elsewhere in your project, it will only work in development.

All of these methods will fail during build:

---
import config from "../../astro.config.mjs";
console.log("Astro config", config);
---

---
const config = await Astro.glob("../../astro.config.mjs");
console.log("Astro config", config[0].default);
---

const config = import.meta.glob("../../astro.config.mjs");

for (const path in config) {
  config[path]().then((mod) => {
    console.log(mod.default);
  });
}

This is due to the use of any Astro imports like @astrojs/react which I guess can't be properly parsed or serialized during build.

One workaround is to create a separate config file that doesn't use any Astro related imports which you import in astro.config.mjs and anywhere else:

export default {
  site: "example.com",
  server: {
    port: 3000,
  },
  // ...
};

import config from "/src/config.ts";
import react from "@astrojs/react";

export default defineConfig({
  ...config,
  integrations: [
    react(),
  ], 
  // ...
});

---
import config from "../config.ts";
console.log("config", config);
---

With this method you won't be able to read the functions you set, but if you wanted to, you could set additional values in the config object (config.ts) which you import in astro.config.mjs, and manually add the functions to the relevant key like integrations, based on which ones exist.

Syntax or rehype plugins

To use a plugin like Rehype Pretty Code, you will need to disable the default Astro syntax highlighting in your astro.config.mjs:

markdown: {
  syntaxHighlight: false
}

Astro's highlighting runs last, so it has priority over any existing plugins.

Feel free to follow me or check out my blog.

Astro loading spinners

Alex Nguyen — Fri, 01 Dec 2023 06:08:16 +0000

Loading spinners with framework components

If you use a framework component with a client directive of client:only, you may see a flash of no content until the component renders.

In this case, it may be useful to show a loading state like a spinner or skeleton until the component loads - especially if it will take longer than a second.

One way to do this is to use a separate loading element alongside your framework component, and hide or remove that element once the component mounts.

React example

<!-- some sort of indicator or spinner --> 
<svg
  class="spinner"
  ...
  >
</svg>
<ReactComponent client:only="react" />

/* ReactComponent.tsx */
import { useEffect } from "react";

export function ReactComponent() {
  useEffect(() => {
    const spinner = document.querySelector(".spinner");
    if (spinner) spinner.remove();
  }, []);

  return <div>React content</div>;
}

Vue example

<VueComponent client:only="vue" />

<!-- VueComponent.vue -->
<script setup lang="ts">
  import { onMounted } from "vue";

  onMounted(() => {
    const spinner = document.querySelector(".spinner");
    if (spinner) spinner.remove();
  });
</script>

<template>
  <div>Vue content</div>
</template>

Svelte example

<SvelteComponent client:only="svelte" />

<!-- SvelteComponent.svelte -->
<script lang="ts">
  import { onMount } from 'svelte';

  onMount(() => {
    const spinner = document.querySelector(".spinner");
    if (spinner) spinner.remove();
  });
</script>

<div>Svelte content</div>

Alternative

If you're using another directive like client:load or client:visible, your component's initial view (before it hydrates) may render right away.

In this case, you probably don't need to do anything. But if you still need to show a loading state, then you can handle this within the component itself. For example, you could add a loading spinner within the component, and then conditionally render the final view once it's ready (with state).

Feel free to follow me or check out my blog.