DEV Community

Discussion on: AI Magic in Atlassian Forge: Local Semantic Search with Forge SQL

View post
Collapse
 
vzakharchenko profile image
Vasiliy Zakharchenko • • Edited

Small addition to this article.

In this example, embeddings are generated in the Custom UI (browser). This works well and keeps the flow fully local.

However, it is also possible to run the embedding model entirely on the Forge backend (inside the resolver), while still staying aligned with Runs on Atlassian eligibility.

This approach has a few practical differences:

  • the frontend sends only plain text
  • embeddings are generated on the backend
  • no model loading required in Custom UI, and no browser-side WebAssembly setup

You can use the same model - Xenova/all-MiniLM-L6-v2 - directly on the backend.

The setup is slightly different:

1. Add the model files into the app

The model files can be placed inside the app like this:

models/
  all-MiniLM-L6-v2/
    config.json
    special_tokens_map.json
    tokenizer.json
    tokenizer_config.json
    onnx/
      model.onnx
Enter fullscreen mode Exit fullscreen mode

One important detail: model_quantized.onnx should be renamed to model.onnx.

2. Include the model files in manifest.yml

The model files need to be included in the Forge package:

app:
  runtime:
    name: nodejs24.x
    architecture: arm64
  id: ari:cloud:ecosystem::app/e05c9f71-3320-4eb7-bf83-89c20c6f8d21
  package:
    extraFiles:
      - models/**/*
Enter fullscreen mode Exit fullscreen mode

Also, backend embeddings do not need the Custom UI wasm setup from the frontend example.

3. Build a separate sidecar bundle with @huggingface/transformers

In my case, I created a separate small library for this:

ai-lib/

Its package.json looks like this:

{
  "name": "ai-lib",
  "version": "1.0.0",
  "type": "commonjs",
  "main": "index.js",
  "scripts": {
    "build:tunnel": "vite build --mode tunnel",
    "build:arm64": "vite build --mode production:arm64",
    "build:x86_64": "vite build --mode production:x86_64"
  },
  "dependencies": {
    "@huggingface/transformers": "^4.0.1",
    "vite": "^8.0.8"
  },
  "devDependencies": {
    "esbuild": "^0.28.0",
    "rollup-plugin-copy": "^3.5.0",
    "vite-plugin-static-copy": "^4.0.1"
  }
}
Enter fullscreen mode Exit fullscreen mode

This is built separately because the default Forge webpack build does not handle the dynamic import path for the model bundle the way we need here.

4. Build for the correct architecture

For forge deploy, the ai-lib bundle must match the runtime architecture defined in manifest.yml.

For example:

app:
  runtime:
    name: nodejs24.x
    architecture: arm64
Enter fullscreen mode Exit fullscreen mode

If architecture is set to arm64, build the sidecar bundle for arm64:

npm run build:arm64
Enter fullscreen mode Exit fullscreen mode

If architecture is set to x86_64, build it for x86_64:

npm run build:x86_64
Enter fullscreen mode Exit fullscreen mode

If architecture is not specified in manifest.yml, use x86_64.

For forge tunnel, use:

npm run build:tunnel
Enter fullscreen mode Exit fullscreen mode

In the tunnel case, the build is intended for local development and depends on your machine architecture.

5. Backend embedding code

The vector generation code can look like this:

import { env, pipeline } from "@huggingface/transformers";
import path from "node:path";

const MODEL_NAME = "all-MiniLM-L6-v2";

export const initAI = async (basePath: string) => {
  env.allowLocalModels = true;
  env.allowRemoteModels = false;

  env.localModelPath = path.join(basePath, "models/");
  env.backends.onnx.wasm!.proxy = false;
  env.backends.onnx.wasm!.wasmPaths = path.join(basePath, "wasm/");

  const extractor = await pipeline("feature-extraction", MODEL_NAME, {
    device: "cpu",
  });

  return {
    async getVector(text: string): Promise<number[]> {
      const output = await extractor(text, { pooling: "mean", normalize: true });
      return Array.from(output.data) as number[];
    },
  };
};
Enter fullscreen mode Exit fullscreen mode

6. Include the built sidecar bundle in manifest.yml

After the bundle is built, include it in the app package too:

app:
  runtime:
    name: nodejs24.x
    architecture: arm64
  id: ari:cloud:ecosystem::app/e05c9f71-3320-4eb7-bf83-89c20c6f8d21
  package:
    extraFiles:
      - models/**/*
      - ai-lib/dist/**/*
Enter fullscreen mode Exit fullscreen mode

7. Load the bundle dynamically on the backend

In the backend code, the built bundle can be loaded dynamically like this:

import path from "node:path";

let aiInstance: any = null;

export const getVector = async (text: string): Promise<number[]> => {
  if (!aiInstance) {
    const sidecarPath = path.join(process.cwd(), "ai-lib/dist/dist/index.mjs");

    try {
      const importDynamic = new Function("modulePath", "return import(modulePath)");
      const module = await importDynamic(sidecarPath);
      const initAI = module.initAI;

      aiInstance = await initAI(process.cwd());
    } catch (err) {
      console.error("Failed to load AI sidecar bundle:", err);
      throw err;
    }
  }

  return await aiInstance.getVector(text);
};
Enter fullscreen mode Exit fullscreen mode

8. Use it directly in the resolver

Then the resolver can generate embeddings on the backend:

import { getVector } from "./ai";

resolver.define("create", async (req: Request<{ data: CreateDocumentInput }>): Promise<number> => {
  const { title, document } = req.payload.data;
  const embedding = await getVector(document);
  const res = await forgeSQL.insert(embeddedDocuments).values([{ title, document, embedding }]);
  return res[0].insertId;
});
Enter fullscreen mode Exit fullscreen mode

With this approach, the frontend sends only plain text, and the vectors for semantic search are computed fully on the backend.

I created a full working example here:

forge-sql-orm-example-backend-ai