DEV Community: Aymeric PINEAU

Automate your JSON translations for i18next / next-intl / vue-i18n

Aymeric PINEAU — Mon, 10 Nov 2025 12:51:22 +0000

If you’ve ever implemented i18next, next-intl, or vue-i18n you probably know that internationalization often slows down your development process.

Why?

You start by declaring your content in your base language. Then you spend time copying it, sending it to your favorite AI provider, waiting for translations, and finally pasting everything back into your /locales or /messages folder.
And you repeat this process for each locale and each namespace.

Whenever your content changes, you go through the same painful loop again.
If your JSON files are too large, you might even have to split them into chunks just to make things work.

To solve this, many teams turn to localization platforms that charge per key, which can get costly for large projects.

In my opinion, translations themselves have no real value anymore.
In 2025, a well-designed script connected to your favorite AI provider can do it better, faster, and cheaper than adding yet another vendor-locked solution to your tech stack.

That’s exactly what Intlayer aims to offer — a script that generates your missing translations at the cost of your chosen AI model.

💡 Example: Adding an additional language to the Intlayer website (100+ English pages) cost around $4 using gpt-5-mini, no need for a larger model.
Want to give it a try? Clone the repo, run the script on one of our 200 missing locales, and submit your PR

What can Intlayer do for you?

Automated testing support – Run Intlayer from your CLI, in your CI/CD pipelines, or even within your unit tests.
Auto-fill missing translations – Intlayer detects missing strings and translates only those.
Smart chunking – If your JSON is large, Intlayer splits it automatically and translates each part independently.
Parallel translation – Handle hundreds of namespaces efficiently with built-in parallelization.
Resilient AI handling – If your AI provider returns inconsistent structures (string vs. object), Intlayer detects, retries, and fixes the issue automatically.
Freedom of choice – Use any AI provider (openAI, Anthropic, DeepSeek, Google, Mistral) with your own API key.

-> Learn more: Intlayer with next-i18next

Building a Smart Documentation, based on OpenAI Embeddings (Chunking, Indexing, and Searching)

Aymeric PINEAU — Wed, 30 Apr 2025 11:04:25 +0000

Hey everyone! I wanted to share my approach to creating a “smart documentation” chatbot for a project I'm working on. I’m not an AI expert, so any suggestions or improvements are more than welcome!

The purpose of this post is not to create another tutorial on building a chatbot based on OpenAI. There's already plenty of content on that topic.
Instead, the main idea is to index documentation, by splitting them into manageable chunks, generating embeddings with OpenAI, and performing a similarity search to find and return the most relevant information to a user's query.

In my case, the documentation will be Markdown files, but it can be any form of text, database object, etc.

Why?

Because it can sometimes be hard to find the information you need, I wanted to create a chatbot that could answer questions about a specific topic, and provide relevant context from the documentation.

This assistant can be used in a variety of ways, such as:

Providing quick answers to frequently asked questions
Searching a doc / page as Algolia does
Helping users find the information they need in a specific doc
Retrieving user worries / questions by storing the asked questions

Summary

Below, I’ll outline the three major parts of my solution:

Reading documentation files
Indexing the documentation (chunking, overlap, and embedding)
Searching the documentation (and hooking it up to a chatbot)

File tree

.
└── docs
    └── ...md
└── src
    └── askDocQuestion.ts
    └── index.ts # Express.js application endpoint
└── embeddings.json # Storage for embeddings
└── packages.json

1. Reading Documentation Files

Instead of hardcoding the documentation text, you can scan a folder for .md files using tools like glob.

// Example snippet of fetching files from a folder:
import fs from "node:fs";
import path from "node:path";
import glob from "glob";

const DOC_FOLDER_PATH = "./docs";

type FileData = {
  path: string;
  content: string;
};

const readAllMarkdownFiles = (): FileData[] => {
  const filesContent: FileData[] = [];
  const filePaths = glob.sync(`${DOC_FOLDER_PATH}/**/*.md`);

  filePaths.forEach((filePath) => {
    const content = fs.readFileSync(filePath, "utf8");
    filesContent.push({ path: filePath, content });
  });

  return filesContent;
};

As an alternative, you can of course fetch your documentation from your database or CMS etc.

2. Indexing the Documentation

To create our search engine, we will use OpenAI's vector embeddings API to generate our embeddings.

Vector embeddings are a way to represent data in a numerical format, which can be used to perform similarity searches (in our case, between the user question, and our documentations sections).

This vector, constituted of a list of floating point numbers, will be used to calculate the similarity using a mathematical formula.

[
  -0.0002630692, -0.029749284, 0.010225477, -0.009224428, -0.0065269712,
  -0.002665544, 0.003214777, 0.04235309, -0.033162255, -0.00080789323,
  //...+1533 elements
];

Based on this concept, was created Vector Database. As a result, instead of using the OpenAI API, it's possible to use a vector database like Chroma, Qdrant or Pinecone.

2.1 Chunk Each File & Overlap

Large blocks of text can exceed model context limits or cause less relevant hits, so it's recommended to split them into chunks to make the search more targeted.
However, to preserve some continuity between chunks, we overlap them by a certain number of tokens (or characters). That way, chunk boundaries are less likely to cut off relevant context mid-sentence.

Example of chunking

In this example, we have a long text that we want to split into smaller chunks.
In this case, we want to create chunks of 100 characters and overlap them by 50 characters.

Full Text (406 characters):

In the heart of the bustling city, there stood an old library that many had forgotten. Its towering shelves were filled with books from every imaginable genre, each whispering stories of adventures, mysteries, and timeless wisdom. Every evening, a dedicated librarian would open its doors, welcoming curious minds eager to explore the vast knowledge within. Children would gather for storytelling sessions.

Chunk 1 (Characters 1-150):

In the heart of the bustling city, there stood an old library that many had forgotten. Its towering shelves were filled with books from every imaginabl.

Chunk 2 (Characters 101-250):

shelves were filled with books from every imaginable genre, each whispering stories of adventures, mysteries, and timeless wisdom. Every evening, a d

Chunk 3 (Characters 201-350):

ysteries, and timeless wisdom. Every evening, a dedicated librarian would open its doors, welcoming curious minds eager to explore the vast knowledge

Chunk 4 (Characters 301-406):

curious minds eager to explore the vast knowledge within. Children would gather for storytelling sessions.

Code Snippet

const CHARS_PER_TOKEN = 4.15; // Approximate pessimistically number of characters per token. Can use `tiktoken` or other tokenizers to calculate it more precisely

const MAX_TOKENS = 500; // Maximum number of tokens per chunk
const OVERLAP_TOKENS = 100; // Number of tokens to overlap between chunks

const maxChar = MAX_TOKENS * CHARS_PER_TOKEN;
const overlapChar = OVERLAP_TOKENS * CHARS_PER_TOKEN;

const chunkText = (text: string): string[] => {
  const chunks: string[] = [];
  let start = 0;

  while (start < text.length) {
    let end = Math.min(start + maxChar, text.length);

    // Don’t cut a word in half if possible:
    if (end < text.length) {
      const lastSpace = text.lastIndexOf(" ", end);
      if (lastSpace > start) end = lastSpace;
    }

    chunks.push(text.substring(start, end));
    // Overlap management
    const nextStart = end - overlapChar;
    start = nextStart <= start ? end : nextStart;
  }

  return chunks;
};

To learn more about chunking, and the impact of the size on the embedding, you can check out this article.

2.2 Embedding Generation

Once a file is chunked, we generate vector embeddings for each chunk using OpenAI’s API (e.g., text-embedding-3-large).

import { OpenAI } from "openai";

const EMBEDDING_MODEL: OpenAI.Embeddings.EmbeddingModel =
  "text-embedding-3-large"; // Model to use for embedding generation

const openai = new OpenAI({ apiKey: OPENAI_API_KEY });

const generateEmbedding = async (textChunk: string): Promise<number[]> => {
  const response = await openai.embeddings.create({
    model: EMBEDDING_MODEL,
    input: textChunk,
  });

  return response.data[0].embedding; // Return the generated embedding
};

2.3 Generating and Save Embeddings for the whole file

To avoid regenerating embeddings every time, we will store the embeddings. It can be store in a database. But in this case, we will simply store it in a JSON file locally.

The following code simply :

iterates over each document,
chunks the document into chunks,
generates embeddings for each chunk,
stores the embeddings in a JSON file.
Fill the vectorStore with the embeddings to be used in the search.

import embeddingsList from "../embeddings.json";

/**
 * Simple in-memory vector store to hold document embeddings and their content.
 * Each entry contains:
 * - filePath: A unique key identifying the document
 * - chunkNumber: The number of the chunk within the document
 * - content: The actual text content of the chunk
 * - embedding: The numerical embedding vector for the chunk
 */
const vectorStore: {
  filePath: string;
  chunkNumber: number;
  content: string;
  embedding: number[];
}[] = [];

/**
 * Indexes all Markdown documents by generating embeddings for each chunk and storing them in memory.
 * Also updates the embeddings.json file if new embeddings are generated.
 */
export const indexMarkdownFiles = async (): Promise<void> => {
  // Retrieve documentations
  const docs = readAllMarkdownFiles();

  let newEmbeddings: Record<string, number[]> = {};

  for (const doc of docs) {
    // Split the document into chunks based on headings
    const fileChunks = chunkText(doc.content);

    // Iterate over each chunk within the current file
    for (const chunkIndex of Object.keys(fileChunks)) {
      const chunkNumber = Number(chunkIndex) + 1; // Chunk number starts at 1
      const chunksNumber = fileChunks.length;

      const chunk = fileChunks[chunkIndex as keyof typeof fileChunks] as string;

      const embeddingKeyName = `${doc.path}/chunk_${chunkNumber}`; // Unique key for the chunk

      // Retrieve precomputed embedding if available
      const existingEmbedding = embeddingsList[
        embeddingKeyName as keyof typeof embeddingsList
      ] as number[] | undefined;

      let embedding = existingEmbedding; // Use existing embedding if available

      if (!embedding) {
        embedding = await generateEmbedding(chunk); // Generate embedding if not present
      }

      newEmbeddings = { ...newEmbeddings, [embeddingKeyName]: embedding };

      // Store the embedding and content in the in-memory vector store
      vectorStore.push({
        filePath: doc.path,
        chunkNumber,
        embedding,
        content: chunk,
      });

      console.info(`- Indexed: ${embeddingKeyName}/${chunksNumber}`);
    }
  }

  /**
   * Compare the newly generated embeddings with existing ones
   *
   * If there is change, update the embeddings.json file
   */
  try {
    if (JSON.stringify(newEmbeddings) !== JSON.stringify(embeddingsList)) {
      fs.writeFileSync(
        "./embeddings.json",
        JSON.stringify(newEmbeddings, null, 2)
      );
    }
  } catch (error) {
    console.error(error);
  }
};

3. Searching the Documentation

3.1 Vector Similarity

To answer a user’s question, we first generate an embedding for the user question and then compute the cosine similarity between the query embedding and each chunk’s embedding. We filter out anything below a certain similarity threshold and keep only the top X matches.

/**
 * Calculates the cosine similarity between two vectors.
 * Cosine similarity measures the cosine of the angle between two vectors in an inner product space.
 * Used to determine the similarity between chunks of text.
 *
 * @param vecA - The first vector
 * @param vecB - The second vector
 * @returns The cosine similarity score
 */
const cosineSimilarity = (vecA: number[], vecB: number[]): number => {
  // Calculate the dot product of the two vectors
  const dotProduct = vecA.reduce((sum, a, idx) => sum + a * vecB[idx], 0);

  // Calculate the magnitude (Euclidean norm) of each vector
  const magnitudeA = Math.sqrt(vecA.reduce((sum, a) => sum + a * a, 0));
  const magnitudeB = Math.sqrt(vecB.reduce((sum, b) => sum + b * b, 0));

  // Compute and return the cosine similarity
  return dotProduct / (magnitudeA * magnitudeB);
};

const MIN_RELEVANT_CHUNKS_SIMILARITY = 0.77; // Minimum similarity required for a chunk to be considered relevant
const MAX_RELEVANT_CHUNKS_NB = 15; // Maximum number of relevant chunks to attach to chatGPT context

/**
 * Searches the indexed documents for the most relevant chunks based on a query.
 * Utilizes cosine similarity to find the closest matching embeddings.
 *
 * @param query - The search query provided by the user
 * @returns An array of the top matching document chunks' content
 */
const searchChunkReference = async (query: string) => {
  // Generate an embedding for the user's query
  const queryEmbedding = await generateEmbedding(query);

  // Calculate similarity scores between the query embedding and each document's embedding
  const results = vectorStore
    .map((doc) => ({
      ...doc,
      similarity: cosineSimilarity(queryEmbedding, doc.embedding), // Add similarity score to each doc
    }))
    // Filter out documents with low similarity scores
    // Avoid to pollute the context with irrelevant chunks
    .filter((doc) => doc.similarity > MIN_RELEVANT_CHUNKS_SIMILARITY)
    .sort((a, b) => b.similarity - a.similarity) // Sort documents by highest similarity first
    .slice(0, MAX_RELEVANT_CHUNKS_NB); // Select the top most similar documents

  // Return the content of the top matching documents
  return results;
};

3.2 Prompting OpenAI with Relevant Chunks

After sorting, we feed the top chunks into the system prompt of the ChatGPT request. This means ChatGPT sees the most relevant sections of your docs as if you had typed them into the conversation. Then we let ChatGPT form an answer for the user.

const MODEL: OpenAI.Chat.ChatModel = "gpt-4o-2024-11-20"; // Model to use for chat completions

// Define the structure of messages used in chat completions
export type ChatCompletionRequestMessage = {
  role: "system" | "user" | "assistant"; // The role of the message sender
  content: string; // The text content of the message
};

/**
 * Handles the "Ask a question" endpoint in an Express.js route.
 * Processes user messages, retrieves relevant documents, and interacts with OpenAI's chat API to generate responses.
 *
 * @param messages - An array of chat messages from the user and assistant
 * @returns The assistant's response as a string
 */
export const askDocQuestion = async (
  messages: ChatCompletionRequestMessage[]
): Promise<string> => {
  // Assistant's response are filtered out otherwise the chatbot will be stuck in a self-referential loop
  // Note that the embedding precision will be lowered if the user change of context in the chat
  const userMessages = messages.filter((message) => message.role === "user");

  // Format the user's question to keep only the relevant keywords
  const formattedUserMessages = userMessages
    .map((message) => `- ${message.content}`)
    .join("\n");

  // 1) Find relevant documents based on the user's question
  const relevantChunks = await searchChunkReference(formattedUserMessages);

  // 2) Integrate the relevant documents into the initial system prompt
  const messagesList: ChatCompletionRequestMessage[] = [
    {
      role: "system",
      content:
        "Ignore all previous instructions. \
        You're an helpful chatbot.\
        ...\
        Here is the relevant documentation:\
        " +
        relevantChunks
          .map(
            (doc, idx) =>
              `[Chunk ${idx}] filePath = "${doc.filePath}":\n${doc.content}`
          )
          .join("\n\n"), // Insert relevant chunks into the prompt
    },
    ...messages, // Include the chat history
  ];

  // 3) Send the compiled messages to OpenAI's Chat Completion API (using a specific model)
  const response = await openai.chat.completions.create({
    model: MODEL,
    messages: messagesList,
  });

  const result = response.choices[0].message.content; // Extract the assistant's reply

  if (!result) {
    throw new Error("No response from OpenAI");
  }

  return result;
};

4. Implement OpenAI API for Chatbot Using Express

To execute our system we will use an Express.js server.
Here an example of a small Express.js endpoint to handle the query:

import express, { type Request, type Response } from "express";
import {
  ChatCompletionRequestMessage,
  askDocQuestion,
  indexMarkdownFiles,
} from "./askDocQuestion";

// Automatically fill the vector store with embeddings when server starts
indexMarkdownFiles();

const app = express();

// Parse incoming requests with JSON payloads
app.use(express.json());

type AskRequestBody = {
  messages: ChatCompletionRequestMessage[];
};

// Routes
app.post(
  "/ask",
  async (
    req: Request<undefined, undefined, AskRequestBody>,
    res: Response<string>
  ) => {
    try {
      const response = await askDocQuestion(req.body.messages);

      res.json(response);
    } catch (error) {
      console.error(error);
    }
  }
);

// Start server
app.listen(3000, () => {
  console.log(`Listening on port 3000`);
});

5. UI: Making a Chatbot Interface

On the frontend, I built a small React component with a chat-like interface. It sends messages to my Express backend and displays the replies. Nothing too fancy, so we’ll skip the details.

Code template

I made a code template for you to use as a starting point for your own chatbot.

Live Demo

If you want a test the final implement this chatbot, check out this demo page

My demo Code

Backend: askDocQuestion.ts
Frontend: ChatBot components

Go Further

On Youtube, have a look at this video of Adrien Twarog as treat of OpenAI Embeddings and Vector Databases

I also stumbled upon OpenAI’s Assistants File Search documentation, which might be interesting if you want an alternative approach.

Conclusion

I hope this gives you an idea of how to handle documentation indexing for a chatbot:

Using chunking + overlap so that the right context is found,
Generating embeddings and storing them for quick vector similarity searches,
Finally handing it off to ChatGPT with the relevant context.

I’m not an AI expert, this is just a solution that I found works well for my needs. If you have any tips on improving efficiency or a more polished approach, please let me know. I’d love to hear feedback on vector storage solutions, chunking strategies, or any other performance tips.

Thanks for reading and feel free to share your thoughts!

A React Native & Lynx i18n solution that helps you keep your translations organized

Aymeric PINEAU — Fri, 21 Mar 2025 15:16:35 +0000

If you're working on making your React Native (or even web) application multilingual, you've probably already tried integrating react-i18next, i18n-js, LinguiJS or other alternatives.

In every project I’ve worked on, the same issues arise:

❌ Unused key-value pairs are never removed.
❌ Content is often duplicated.
❌ Ensuring format consistency across all languages and verifying that each translation is present and accurate becomes challenging, especially when managing more than five locale directories locally.
❌ Even if third-party tools can to solve this problem, by default i18next doesn’t generate TypeScript types, which means you can reference a key like t("my.key") even if it has been deleted.
❌ Additionally, localization platforms like Localize, Lokalise, or Locize can quickly become costly.

Tired of this complexity, I started looking for a solution to address these problems. I waited, and waited… before finally developing Intlayer.

✨ Key Features of Intlayer

✅ Available for React Native and Lynx
✅ Simple and quick integration

✅ Content declaration in the same directory as your component (or everywhere in your project)

✅ Autogenerated TypeScript Types – No more guessing or runtime errors due to missing keys

✅ Content declaration in either JSON, JS, or TS format

✅ Allows embedding external files (Markdown, TXT, etc.)
✅ Instantly fetch external data with automatic typing

✅ Intlayer natively provides a way to externalize your content and make it editable via a CMS

⚡ Getting Started (React Native)

1️⃣ Install the Required Packages

Run the following command in your project:

npm install intlayer react-intlayer react-native-intlayer

2️⃣ Configure Your Locales

Create an intlayer.config.ts file in your project root:

import { Locales, type IntlayerConfig } from "intlayer";

const config: IntlayerConfig = {
  internationalization: {
    locales: [Locales.ENGLISH, Locales.FRENCH, Locales.SPANISH],
    defaultLocale: Locales.ENGLISH,
  },
};

export default config;

3️⃣ Enable Metro Support

Add the Metro bundler plugin to your metro.config.js:

const { getDefaultConfig } = require("expo/metro-config");
const { configMetroIntlayer } = require("react-native-intlayer/metro");

module.exports = (async () => {
  const defaultConfig = getDefaultConfig(__dirname);
  return await configMetroIntlayer(defaultConfig);
})();

4️⃣ Wrap Your App with the Intlayer Provider

Modify your _layout.tsx file to initialize Intlayer and detect the user's language:

import { Stack } from "expo-router";
import { getLocales } from "expo-localization";
import { IntlayerProviderContent } from "react-intlayer";
import { intlayerPolyfill } from "react-native-intlayer";

intlayerPolyfill();

const getDeviceLocale = () => getLocales()[0]?.languageTag;

const RootLayout = () => {
  return (
    <IntlayerProviderContent defaultLocale={getDeviceLocale()}>
      <Stack>
        <Stack.Screen name="(tabs)" />
      </Stack>
    </IntlayerProviderContent>
  );
};

export default RootLayout;

5️⃣ Define Translations Close to Your Components

Instead of maintaining large JSON files, define your translations inside the component's directory:

import { t, md, file, type Dictionary } from "intlayer";

const homeScreenContent = {
  key: "home-screen",
  content: {
    title: t({
      en: "My Title",
      fr: "Mon titre",
      es: "Mi título",
    }),
    description: t({
      en: md(file("./myDescription.en.md")),
      fr: md(file("./myDescription.fr.md")),
      es: md(file("./myDescription.es.md")),
    }),
    contentFetch: fetch("https://example.com").then((res) => res.text()),
  },
} satisfies Dictionary;

export default homeScreenContent;

Now, you can use translations inside your components like this:

import { Text, View } from "react-native";
import { useIntlayer } from "react-intlayer";

const MyComponent = () => {
  const { title, description, contentFetch } = useIntlayer("my-component");

  return (
    <View>
      <Text>{title}</Text>
      <Text>{description}</Text>
      <Text>{contentFetch}</Text>
    </View>
  );
};

🔄 Switching Languages Dynamically

To allow users to change languages at runtime, use the useLocale hook:

import { View, Text, TouchableOpacity } from "react-native";
import { getLocaleName } from "intlayer";
import { useLocale } from "react-intlayer";

const LocaleSwitcher = () => {
  const { setLocale, availableLocales } = useLocale();

  return (
    <View>
      {availableLocales.map((locale) => (
        <TouchableOpacity key={locale} onPress={() => setLocale(locale)}>
          <Text>{getLocaleName(locale)}</Text>
        </TouchableOpacity>
      ))}
    </View>
  );
};

export default LocaleSwitcher;

Resources

⭐ GitHub issues & feedback: Intlayer Repository

📄 Docs: React Native & Expo Docs

🔹 Template: React Native Template

A Visual Editor for Your Website – Free & Open Source!

Aymeric PINEAU — Wed, 19 Mar 2025 13:50:57 +0000

Do you have content on your website? Ever wished you could edit it visually, without digging into code?

Intlayer Visual Editor, a free and open-source tool that allows you to edit your web app’s content directly in a visual interface.

Why Intlayer Visual Editor?

Multilingual support – Manage content in multiple languages effortlessly.
Works with Markdown – Perfect for static sites and developers.
Handles external files – Supports .md, .txt, and more.
Open Source & Free – Modify and extend it as needed.

Try it out: https://intlayer.org/playground

Have issues or feature ideas? GitHub Repo

How It Works

The visual editor consists of:

A frontend application – Loads your website inside an iframe and detects editable content.
A server-side process – When you save changes, it updates your content declaration files (in JSON format for now).

Installation & Setup

Once Intlayer is configured in your project, install intlayer-editor:

npm install intlayer-editor --save-dev

Then, configure a intlayer.config.ts file:

const config = {
  editor: {
    applicationURL: "http://localhost:3000",
  },
};
export default config;

Using the Editor

Once installed, start the editor:

npx intlayer-editor start

Then, open http://localhost:8000 in your browser and hover over content to edit it! 🎨

Get Involved!

This project is 100% open-source! If you want to contribute, report issues, or suggest features:

GitHub Repo: https://github.com/aymericzip/intlayer

Would love to hear your thoughts and feedback! Let’s make content editing easier for developers.

Showcase of a Lightweight Hook for Async Data Fetching & Caching in React

Aymeric PINEAU — Tue, 07 Jan 2025 10:17:12 +0000

Hey everyone!

I’ve been working on a lightweight React hook that mimics some of the essential features of React Query (like fetching, caching, retries, etc.) but in a more compact, easily customizable package. Below is a quick breakdown of how it works internally, referencing the relevant code sections. If you want to see the entire code, head over to the repo:

Full Source Code on GitHub.
The hook is also available on npm as api-refetch.

Why Making My Own Hook?

While React Query and SWR are both great libraries, I wanted a more hands-on approach for a few reasons:

Lightweight Footprint

While React Query and SWR are feature-rich, they can be relatively large. Making your own hook would be ideal when package size is a big concern. This hook as meant to be installed as a dependency of another library (Intlayer). As a result, the size of the solution was an important consideration.
Easy to Customize & Optimize
I needed some specific capabilities—like storing/fetching data from local storage and managing parallel requests using a straightforward approach.
By cloning the repo or copying the code directly into your project, you can strip out any unwanted features and keep only what you need. This not only reduces bundle size but also minimizes unnecessary re-renders and increase, giving you a leaner, more performant solution tailored to your specific requirements.
No Required Provider

I wanted to avoid Context Provider to make the hook global, and keep it's usage as simple as possible. So I made a version of the hook based on a Zustand store (see example bellow).
Learning Exercise

Building an async library from scratch is an excellent way to understand concurrency, caching, and state management internals.

In short, rolling my own hook was a chance to hone in on precisely the features I need (and skip the ones I don’t) while keeping the library small and easy to understand.

Covered Features

The React hook manage:

Fetching & State Management: Handles loading, error, success, and fetched states.
Caching & Storage: Optionally caches data (via React states or Zustand under the hood) and offers local storage support.
Retries & Revalidation: Configurable retry limits and automatic revalidation intervals.
Activation & Invalidation: Automatically activates and invalidates queries depending on other queries' or states. Example: automatically fetch some data when the user logs in. And invalidate it when the user logs out.
Parallel Component Mount Fetching: Prevents multiple simultaneous requests for the same resource when multiple components mount at once.

How the Code Works

Below are the key points in api-refetch and short references to the relevant parts of the code in useAsync.tsx.

1. Fetching and Handling Parallel Mounting

Code Snippet:

  // This map stores any in-progress Promise to avoid sending parallel requests
  // for the same resource across multiple components.
  const pendingPromises = new Map();

  const fetch: T = async (...args) => {
    // Check if a request with the same key + args is already running
    if (pendingPromises.has(keyWithArgs)) {
      return pendingPromises.get(keyWithArgs);
    }

    // Otherwise, store a new Promise and execute
    const promise = (async () => {
      setQueryState(keyWithArgs, { isLoading: true });

      // ...perform fetch here...
    })();

    // Keep it in the map until it resolves or rejects
    pendingPromises.set(keyWithArgs, promise);
    return await promise;
  };

Explanation: Here, we store any ongoing fetch in a pendingPromises map. When two components try to fetch the same resource simultaneously (by having the same keyWithArgs), the second one just reuses the ongoing request instead of making a duplicate network call.

2. Revalidation

Code Snippet:

  // Handle periodic revalidation if caching is enabled
  useEffect(
    () => {
      if (!revalidationEnabled || revalidateTime <= 0) return; // Revalidation is disabled
      if (!isEnabled || !enabled) return; // Hook is disabled
      if (isLoading) return; // Fetch is already in progress
      if (!isSuccess || !fetchedDateTime) return; // Should retry either of revalidate
      if (!(cacheEnabled || storeEnabled)) return; // Useless to revalidate if caching is disabled

      const timeout = setTimeout(() => {
        fetch(...storedArgsRef.current);
      }, revalidateTime);

      return () => clearTimeout(timeout);
    },
    [
      /* dependencies */
    ]
  );

Explanation: Whenever you enable revalidation, api-refetch checks if the cached data is older than a specified revalidateTime. If it is, the data is automatically re-fetched in the background to keep your UI in sync without extra manual triggers.

3. Retry Logic

Code Snippet:

  useEffect(
    () => {
      const isRetryEnabled = errorCount > 0 && retryLimit > 0;
      const isRetryLimitReached = errorCount > retryLimit;

      if (!isEnabled || !enabled) return; // Hook is disabled
      if (!isRetryEnabled) return; // Retry is disabled
      if (isRetryLimitReached) return; // Retry limit has been reached
      if (!(cacheEnabled || storeEnabled)) return; // Useless to retry if caching is disabled
      if (isLoading) return; // Fetch is already in progress
      if (isSuccess) return; // Hook has already fetched successfully

      const timeout = setTimeout(() => {
        fetch(...storedArgsRef.current);
      }, retryTime);

      return () => clearTimeout(timeout);
    },
    [
      /* dependencies */
    ]
  );

Explanation: On error, the hook counts how many failed attempts have occurred. If it’s still below the retryLimit, it automatically waits retryTime milliseconds before trying again. This process continues until data is successfully fetched or the retry limit is reached.

4. Auto-Fetch

Code Snippet:

  // Auto-fetch data on hook mount if autoFetch is true
  useEffect(
    () => {
      if (!autoFetch) return; // Auto-fetch is disabled
      if (!isEnabled || !enabled) return; // Hook is disabled
      if (isFetched && !isInvalidated) return; // Hook have already fetched or invalidated
      if (isLoading) return; // Fetch is already in progress

      fetch(...storedArgsRef.current);
    },
    [
      /* dependencies */
    ]
  );

Explanation: With autoFetch set to true, the hook will automatically run the async function as soon as the component mounts—perfect for “fire-and-forget” scenarios where you always want the data on load.

See the Full Source on GitHub

Check out the complete code, which includes local storage logic, query invalidation, and more here:

Full Source Code

Feel free to give it a try, report issues, or contribute if you’re interested. Any feedback is much appreciated!

Example of use

Installation

Copy the code or code the repo

npm install api-refetch

Quick Example

// 1. Wrap your app in the provider (optional but recommended)
import { AsyncStateProvider, useAsync } from "api-refetch";
// import { useAsync } from "api-refetch/zustand"; // alternatively, use the zustand based hook (do not need the provider)

function App() {
  return (
    <AsyncStateProvider>
      <UserDetails />
    </AsyncStateProvider>
  );
}

// 2. Define the asynchronous function that fetches your data
const fetchUserData = async (): Promise<string> => {
  // Create a Promise that resolves after 1s
  return new Promise((resolve) => {
    setTimeout(() => {
      resolve({
        name: "John Doe",
      });
    }, 1000);
  });
};

// 3. Use the hook in your component
const UserDetails = () => {
  const { isLoading, data, error, revalidate } = useAsync(
    "userDetails",
    async () => await fetchUserData(),
    {
      enable: true, // enable the hook
      cache: true, // cache the API call result using zustand
      store: true, // store the API call result in the local storage
      retryLimit: 3, // retry 3 times if the API call fails
      retryTime: 10 * 1000, // wait 10 seconds before retrying
      autoFetch: true, // auto fetch the API call when the component is mounted
      revalidation: true, // enable revalidation
      revalidateTime: 5 * 60 * 1000, // revalidate every 5 minutes
      isInvalidated: false, // determine if the data is invalidated and should be refetched
      invalidateQueries: ["user"], // invalidate other queries when the data is updated
      updateQueries: ["user"], // set other queries data when the data is updated
      onSuccess: (data) => console.log("User data fetched successfully:", data),
      onError: (error) => console.error("Error fetching user data:", error),
    }
  );

  if (isLoading) return <div>Loading...</div>;
  if (error) return <div>Error: {error}</div>;
  return (
    <div>
      <h1>{data?.name}</h1>
      <button onClick={() => revalidate()}>Refresh</button>
    </div>
  );
};

That’s it! Give it a try, and let me know how it goes. Feedback, questions, or contributions are more than welcome on GitHub.

GitHub: api-refetch

Happy coding!

Translate Your Express Backend API with express-intlayer (i18n)

Aymeric PINEAU — Tue, 10 Dec 2024 11:40:12 +0000

I recently needed to add multilingual support to an Express-based API. I thought I’d share a brief tutorial for anyone looking to make their backend respond with translated content depending on the user’s preferred language.

Why Internationalize Your Backend?

Here are some practical use cases for backend internationalization:

Displaying Backend Errors in User's Language: Show error messages in the user's native language to improve understanding and reduce frustration. It allows errors to be displayed directly in a toast without mapping them on the frontend.
Sending Multilingual Emails, Push Notifications, etc: Increase engagement by sending transactional, marketing, or notification emails in the recipient’s language. Localize Push notification, SMS messages to ensure clarity and enhance user experience.
Retrieving Multilingual Content: Serve content from your database in multiple languages.

Getting Started

Installation

First, install the necessary packages using your preferred package manager:

npm install intlayer express-intlayer

Setup

Next, configure the internationalization settings by creating an intlayer.config.ts file in your project root:

// intlayer.config.ts
import { Locales, type IntlayerConfig } from "intlayer";

const config: IntlayerConfig = {
  internationalization: {
    locales: [
      Locales.ENGLISH,
      Locales.FRENCH,
      Locales.SPANISH_MEXICO,
      Locales.SPANISH_SPAIN,
    ],
    defaultLocale: Locales.ENGLISH,
  },
};

export default config;

Express Application Setup

Now, set up your Express application to use express-intlayer:

// src/index.ts
import express, { type Express } from "express";
import { intlayer, t } from "express-intlayer";

const app: Express = express();

// Load internationalization request handler
app.use(intlayer());

// Routes
app.get("/", (_req, res) => {
  res.send(
    t({
      en: "Example of returned content in English",
      fr: "Exemple de contenu renvoyé en français",
      "es-ES": "Ejemplo de contenido devuelto en español (España)",
      "es-MX": "Ejemplo de contenido devuelto en español (México)",
    })
  );
});

// Start server
app.listen(3000);

Compatibility

express-intlayer works seamlessly with react-intlayer or next-intlayer. But it's also compatible with various internationalization solutions.

Customizing Locale Detection

By default, express-intlayer uses the Accept-Language header to determine the client's preferred language. You can customize this behavior to detect locales through headers or cookies:

import { Locales, type IntlayerConfig } from "intlayer";

const config: IntlayerConfig = {
  // Other configuration options
  middleware: {
    headerName: "my-locale-header",
    cookieName: "my-locale-cookie",
  },
};

Feel free to check out the official documentation for more advanced configurations and features and the official website of the solution.
Here the Github of the solution

Internationalising your React/NextJS application has never been easier since Intlayer

Aymeric PINEAU — Mon, 22 Jul 2024 18:55:56 +0000

Intlayer is an intlayernationalisation solution that allow the declaration of your content everywhere in your code. It converts declaration of multilingual content to integrate easily in your code.

Intlayer addresses:

Solution covering Vite+React, Create React App, or NextJS environement
Management of Server Components
TypeScript validation and autocompletion support

For more details: Interest of Intlayer

Example of usage

.
├── ClientComponent
│   ├── index.content.ts
│   └── index.tsx
└── ServerComponent
    ├── index.tsx
    └── index.content.ts

// ./ClientComponent/index.content.ts

import { DeclarationContent, t } from "intlayer";

const clientComponentContent: DeclarationContent = {
  key: "client-component",
  content: {
    myTranslatedContent: t({
      en: "Hello World",
      fr: "Bonjour le monde",
      es: "Hola Mundo",
    })
  }
};

export default clientComponentContent;

// ./ClientComponent/index.tsx
"use client";

import { useIntlayer } from "next-intlayer";

export const ClientComponent = () => {
  const { myTranslatedContent } = useIntlayer("client-component");

  return <span>{myTranslatedContent}</span>;
};

Why Choose Intlayer?

Intlayer integrates better into the React ecosystem, making it possible to declare your content at the same level as your component. This way, you can easily copy your code between your applications without having to search for the dictionaries related to that code.

While the other internationalisation solutions require writing JSON dictionaries for each language available for your application, Intlayer simplifies this by declaring the content through a simple JavaScript file.

Using the power of TypeScript, you can also be sure that no translations are missing in your declaration files.

How to integrate Intlayer

To see how to integrate Intlayer into your application, I invite you to consult the online documentation corresponding to your environment:

For each of these environments, setting up your application will take no more than 10 minutes.

Once your content is written in your declaration file, a simple hook allows you to use your content in your components.

And that’s it! Your content will be automatically translated according to what has been declared in your declaration files.

Test it by yourself

Test it live with this interactive online example on CodeSandbox.

Support the projet

To share your comments, bugs, or suggestions: