Semantic Search with TypeScript: Using embed() and embedMany() for Vector Search

In the age of information overload, keyword-based search often falls short. Users aren't just looking for exact matches; they're looking for meaning. This is where semantic search shines, allowing systems to understand the intent behind a query and retrieve results that are conceptually similar, even if they don't contain the exact keywords.

At the heart of semantic search lies the concept of embeddings – dense numerical representations of text that capture its meaning. NeuroLink, the universal AI SDK for TypeScript, simplifies the process of generating and utilizing these embeddings, making it straightforward to build powerful semantic search capabilities into your applications.

This article will guide you through generating embeddings with NeuroLink's embed() and embedMany() methods, performing similarity search, and integrating with vector databases to build a complete semantic search engine.

What are Embeddings and Why Are They Crucial for Semantic Search?

Imagine mapping every word, sentence, or document into a multi-dimensional space where items with similar meanings are located closer to each other. That's essentially what an embedding model does. Each piece of text is transformed into a fixed-size vector (a list of numbers) that encapsulates its semantic properties.

For semantic search, this means:

• Understanding Context: A search for "car repair" can return results about "automobile maintenance" or "vehicle servicing" even if the exact phrase isn't present.
• Ranking Relevance: Results can be ranked based on their semantic similarity to the query, providing more relevant outcomes.
• Bridging Vocabulary Gaps: It overcomes issues arising from synonyms, paraphrases, or different ways of expressing the same idea.

NeuroLink provides a unified API to generate these crucial vectors from various leading AI providers.

Generating Embeddings with NeuroLink: embed() and embedMany()

NeuroLink offers two primary methods for generating embeddings via its ProviderFactory interface: embed() for single text strings and embedMany() for efficient batch processing.

provider.embed(text, modelName?): Single Text Embedding

The embed() method takes a single string of text and returns its corresponding embedding vector.

import { ProviderFactory } from "@juspay/neurolink";

async function getEmbedding(text: string) {
  // Create an OpenAI provider instance for embedding.
  // NeuroLink supports OpenAI, Google AI Studio, Google Vertex, and Amazon Bedrock for embeddings.
  const provider = await ProviderFactory.createProvider("openai");

  // Generate the embedding vector
  const vector = await provider.embed(text);

  console.log(`Text: "${text}"`);
  console.log(`Embedding dimension: ${vector.length}`);
  // console.log("Embedding vector (first 5 elements):", vector.slice(0, 5));

  return vector;
}

// Example usage
const queryEmbedding = await getEmbedding("How do I reset my password?");
const documentEmbedding = await getEmbedding("Troubleshooting password issues");

Key Parameters:

• text (string, required): The input text to be embedded.
• modelName (string, optional): Allows you to override the default embedding model for the chosen provider. For example, text-embedding-3-small for OpenAI or gemini-embedding-001 for Google AI Studio.

provider.embedMany(texts, modelName?): Batch Embedding for Efficiency

For scenarios involving multiple documents or a large corpus, embedMany() is significantly more efficient. It accepts an array of text strings and returns an array of corresponding embedding vectors. NeuroLink (via Vercel AI SDK) intelligently handles batching for providers that have batch size limits.

import { ProviderFactory } from "@juspay/neurolink";

async function getManyEmbeddings(texts: string[]) {
  const provider = await ProviderFactory.createProvider("googleAiStudio");

  // Generate embeddings for multiple texts in a single API call
  const embeddings = await provider.embedMany(texts);

  console.log(`Generated ${embeddings.length} embeddings.`);
  embeddings.forEach((emb, index) => {
    console.log(`Embedding ${index + 1} dimension: ${emb.length}`);
  });

  return embeddings;
}

// Example usage with multiple document snippets
const documents = [
  "NeuroLink provides a unified API for 13+ AI providers.",
  "Semantic search helps find documents by meaning, not just keywords.",
  "Event-driven AI applications can leverage lifecycle hooks for analytics.",
];

const documentEmbeddings = await getManyEmbeddings(documents);

Key Parameters:

• texts (string[], required): An array of text strings to be embedded.
• modelName (string, optional): Same as embed(), allows overriding the default embedding model.

Supported Providers and Model Selection

NeuroLink integrates with several top-tier embedding providers:

• OpenAI: Uses models like text-embedding-3-small (default) or text-embedding-3-large.
• Google AI Studio: Uses gemini-embedding-001 (default).
• Google Vertex: Uses text-embedding-004 (default).
• Amazon Bedrock: Uses models like amazon.titan-embed-text-v2:0 (default).

You can configure the default models using environment variables (e.g., OPENAI_EMBEDDING_MODEL, VERTEX_EMBEDDING_MODEL) or directly within the embed()/embedMany() calls.

For providers that do not natively support embeddings (e.g., Anthropic, Mistral), you can still use NeuroLink for text generation and then use a separate NeuroLink provider instance configured for an embedding-capable provider to handle your embedding needs.

import { ProviderFactory } from "@juspay/neurolink";

// Use Anthropic for chat generation
const chatProvider = await ProviderFactory.createProvider("anthropic");
const response = await chatProvider.generate({
  input: { text: "Tell me a story about a wizard." },
});

// Use OpenAI for embeddings, independently
const embedProvider = await ProviderFactory.createProvider("openai");
const storyEmbedding = await embedProvider.embed(response.text);

Building a Semantic Search Engine: From Embeddings to Vector Databases

Once you have embeddings, the next step is to store them and perform similarity searches. This typically involves a vector database (or vector store), which is optimized for storing and querying high-dimensional vectors.

The general workflow for building a semantic search engine looks like this:

1. Index Documents:

   • Take your corpus of documents (e.g., articles, product descriptions, support tickets).
   • Chunk them into manageable segments if they are large.
   • Use embedMany() to generate embeddings for each segment.
   • Store these embeddings, along with their original text and any metadata, in a vector database.

2. Query and Retrieve:

   • When a user submits a query, use embed() to generate an embedding for the query.
   • Query the vector database to find document embeddings that are most "similar" to the query embedding. Similarity is usually calculated using distance metrics like cosine similarity.
   • Retrieve the original text segments corresponding to the most similar embeddings.

Practical Code Example: In-Memory Semantic Search

Let's illustrate with a basic in-memory vector store, focusing on the core embedding and similarity logic. For production systems, you would integrate with dedicated vector databases like Pinecone, Weaviate, or ChromaDB.

import { NeuroLink, ProviderFactory, InMemoryVectorStore } from "@juspay/neurolink";

// 1. Prepare your documents
const articles = [
  { id: "article1", text: "NeuroLink simplifies AI integration across 13 providers." },
  { id: "article2", text: "Vector embeddings enable semantic search by capturing meaning." },
  { id: "article3", text: "Lifecycle hooks in NeuroLink's middleware system manage AI events." },
  { id: "article4", text: "Building reactive applications with event-driven AI architectures." },
  { id: "article5", text: "The importance of cost tracking and analytics in AI applications." },
];

async function buildSemanticSearchEngine() {
  const embedProvider = await ProviderFactory.createProvider("openai");
  const vectorStore = new InMemoryVectorStore(); // For simplicity, use in-memory

  // 2. Generate embeddings and index documents
  console.log("Indexing documents...");
  const textsToEmbed = articles.map((a) => a.text);
  const embeddings = await embedProvider.embedMany(textsToEmbed);

  const documentNodes = articles.map((article, index) => ({
    id: article.id,
    vector: embeddings[index],
    metadata: { text: article.text },
  }));
  await vectorStore.upsert("docs_index", documentNodes); // Store embeddings in an index
  console.log("Documents indexed successfully.");

  // Function to perform semantic search
  async function semanticSearch(query: string, topK: number = 2) {
    console.log(`Searching for: "${query}"`);
    const queryVector = await embedProvider.embed(query);

    // 3. Query the vector store for similar documents
    const searchResults = await vectorStore.query("docs_index", queryVector, topK);

    console.log("Top results:");
    searchResults.forEach((result, i) => {
      console.log(
        `  ${i + 1}. Score: ${result.score.toFixed(4)}, Text: "${result.metadata.text}"`
      );
    });
    return searchResults;
  }

  // 4. Test the semantic search engine
  await semanticSearch("How to connect to multiple AI services?");
  await semanticSearch("Tell me about application events in AI.");
}

buildSemanticSearchEngine().catch(console.error);

Integration with RAG Pipelines

NeuroLink's rag feature simplifies the entire process by internally handling document chunking, embedding generation, and similarity search. When you use rag: { files } in neurolink.generate() or neurolink.stream(), it transparently leverages embed() and embedMany() under the hood to build context for your AI model.

import { NeuroLink } from "@juspay/neurolink";

const neurolink = new NeuroLink();

// NeuroLink automatically handles embedding and retrieval for RAG
const result = await neurolink.generate({
  input: { text: "What are the benefits of event-driven AI?" },
  rag: {
    files: ["./my-event-driven-ai-guide.md", "./middleware-patterns.pdf"], // Your documents
    chunkSize: 512,
    topK: 3,
  },
});

console.log(result.text); // AI response informed by semantic search of your files

For advanced use cases requiring more control over the embedding and retrieval steps, NeuroLink allows you to use createVectorQueryTool with explicit embed() calls and your chosen vector store.

Conclusion

Semantic search is a game-changer for building intelligent applications, moving beyond keyword matching to true understanding. NeuroLink, with its powerful embed() and embedMany() methods, makes it incredibly simple to integrate embedding generation into your TypeScript projects. Whether you're building a sophisticated RAG pipeline or a standalone semantic search engine, NeuroLink provides the tools to unlock the full potential of vector search. By leveraging these capabilities, you can build AI applications that are not just smart, but truly intuitive and responsive to user intent.

---

NeuroLink — The Universal AI SDK for TypeScript

• GitHub: github.com/juspay/neurolink
• Install: npm install @juspay/neurolink
• Docs: docs.neurolink.ink
• Blog: blog.neurolink.ink — 150+ technical articles