Prosper Otemuyiwa for Valyu AI

Posted on Oct 10

How to Build a Competitive Intelligence Agent for Public Companies

#ai #llm #agents #automation

Day 10 of 30 Days of Search.

Most competitor research breaks in the same place: it finds a few recent headlines, asks a model to summarize them, and forgets where the claims came from.

That is not enough for public-company competitive intelligence. A competitive intelligence agent is a retrieval-and-synthesis workflow that gathers competitor evidence, extracts strategic signals, and returns a cited brief for human review. Public companies leave trails across SEC filings, earnings commentary, investor pages, product pages, pricing pages, news, market data, and analyst commentary where licensed or publicly available. A useful agent should retrieve those sources, normalize them, compare competitors by theme, and preserve citations for every claim it makes.

In this tutorial, we will build a TypeScript competitive intelligence agent for public companies. It will:

identify a peer set for a target company
retrieve SEC filing evidence, news, web context, and market signals
extract competitive moves into a structured table
generate a cited briefing
add a Deep Research mode like the Valyu competitor analysis app

The workflow is:

Target company → peer set → evidence retrieval → signal extraction → cited brief → optional Deep Research report.

TL;DR

Build a TypeScript competitive intelligence agent that compares public companies using SEC filings, news, web evidence, and market context.
Keep retrieval separate from reasoning so every competitive signal cites source IDs.
Use a lightweight brief mode for recurring monitoring.
Escalate to Valyu Deep Research when you need a longer cited report, PDF output, and multi-step synthesis.
This is a research workflow, not investment advice.

What should a public-company competitive intelligence agent track?

A public company gives you several evidence layers:

Evidence layer	Why it matters
SEC filings and EDGAR records	Strategy, risks, segments, acquisitions, customer concentration, and regulatory disclosures
Earnings calls and investor materials	Management's current narrative and guidance
News and web sources	Product launches, partnerships, executive changes, incidents, and market moves
Market data	Price action, valuation context, revenue expectations, and peer movement
Company websites	Product positioning, pricing pages, customer claims, and vertical focus

The agent should not answer “is this company a good investment?” That crosses into investment advice. It should answer narrower research questions such as:

Which competitors are attacking the same customer segment?
What changed in the competitive landscape this week?
Which claims are supported by SEC filings versus news coverage?
Which moves appear material enough to investigate further?

Step 1: set up the TypeScript project

Use Node.js 22 or newer and a Valyu API key.

npm init -y
npm install valyu-js@2.10.3
npm install --save-dev tsx@4.23.15

Create .env:

VALYU_API_KEY=replace-with-your-valyu-key

Keep .env out of version control. The finance datasets you can query depend on your Valyu account and plan. The SEC filing examples use Valyu's SEC filing source; for market-data and premium financial sources, confirm access in the finance guide and pricing.

Create competitive-intel-agent.ts and add the imports and shared types:

import { writeFile } from "node:fs/promises";
import { Valyu } from "valyu-js";

type EvidenceKind = "filing" | "news" | "market" | "web";

type Evidence = {
  id: number;
  kind: EvidenceKind;
  company: string;
  title: string;
  url: string;
  date: string | null;
  text: string;
};

type CompetitiveSignal = {
  company: string;
  theme: "product" | "pricing" | "customers" | "partnership" | "financial" | "risk" | "strategy";
  claim: string;
  whyItMatters: string;
  sourceIds: number[];
  confidence: "low" | "medium" | "high";
};

type Brief = {
  target: string;
  peers: string[];
  summary: string;
  signals: CompetitiveSignal[];
  nextQuestions: string[];
};

const MAX_TEXT_PER_SOURCE = 2_000;
const MAX_TOTAL_EVIDENCE_CHARS = 48_000;
const MAX_SOURCES_PER_COMPANY = 10;

Step 2: define the target and peer set

A competitive intelligence agent needs scope. Start with one target company, then name peers explicitly or ask the retrieval layer to discover them.

const input = {
  target: process.argv[2] ?? "NVIDIA",
  peers: (process.argv[3] ?? "AMD, Intel, Broadcom").split(",").map(s => s.trim()).filter(Boolean),
  focus: process.argv[4] ?? "AI data center accelerators, inference chips, and enterprise AI infrastructure",
};

function companyList(target: string, peers: string[]) {
  return [target, ...peers].filter((value, index, arr) => arr.indexOf(value) === index);
}

For production, store canonical company identifiers: ticker, CIK, exchange, website, and known subsidiaries. Names alone can collide. “Apple” the public company is not every source that mentions apples.

Step 3: retrieve filings, news, market, and web evidence

We will keep retrieval separate from reasoning. Each result becomes an Evidence record with an ID the model must cite.

async function searchValyu(query: string, kind: EvidenceKind, company: string): Promise<Evidence[]> {
  const valyu = new Valyu();
  const options =
    kind === "filing"
      ? {
          searchType: "proprietary" as const,
          includedSources: ["valyu/valyu-sec-filings"],
          maxNumResults: 4,
          responseLength: 9000,
        }
      : kind === "market"
        ? {
            searchType: "proprietary" as const,
            includedSources: ["finance"],
            maxNumResults: 4,
            responseLength: 9000,
          }
        : {
            searchType: "web" as const,
            maxNumResults: 4,
            responseLength: 9000,
          };

  const response = await valyu.search(query, options);
  if (!response.success) throw new Error(`${kind} search failed for ${company}`);

  return response.results
    .filter(result => typeof result.content === "string" && result.content.trim().length > 0)
    .map((result, index) => ({
      id: index + 1,
      kind,
      company,
      title: result.title || `${company} ${kind} result`,
      url: result.url,
      date: result.publication_date ?? null,
      text: String(result.content).replace(/\s+/g, " ").trim().slice(0, MAX_TEXT_PER_SOURCE),
    }));
}

function evidenceBudget(rows: Evidence[]): Evidence[] {
  const seen = new Set<string>();
  const perCompany = new Map<string, number>();
  let totalChars = 0;
  const kept: Evidence[] = [];

  for (const row of rows) {
    const key = `${row.kind}:${row.url}`;
    if (seen.has(key)) continue;

    const count = perCompany.get(row.company) ?? 0;
    if (count >= MAX_SOURCES_PER_COMPANY) continue;
    if (totalChars + row.text.length > MAX_TOTAL_EVIDENCE_CHARS) break;

    seen.add(key);
    perCompany.set(row.company, count + 1);
    kept.push({ ...row, id: kept.length + 1 });
    totalChars += row.text.length;
  }

  return kept;
}

async function collectEvidence(target: string, peers: string[], focus: string): Promise<Evidence[]> {
  const allCompanies = companyList(target, peers);
  const evidence: Evidence[] = [];

  for (const company of allCompanies) {
    const searches: Array<[EvidenceKind, string]> = [
      ["filing", `${company} latest 10-K 10-Q competition strategy risk factors ${focus}`],
      ["news", `${company} recent product launch partnership pricing customer news ${focus}`],
      ["market", `${company} market data revenue segment guidance valuation competitors ${focus}`],
      ["web", `${company} official product pricing customer pages ${focus}`],
    ];

    for (const [kind, query] of searches) {
      evidence.push(...await searchValyu(query, kind, company));
    }
  }

  const scoped = evidenceBudget(evidence);
  if (!scoped.length) throw new Error("No evidence returned. Narrow the company, ticker, or focus.");
  return scoped;
}

This deliberately mixes slow-moving disclosures with fast-moving news. The agent can then tell you whether a competitive signal is durable, recent, or still speculative.

In production, do not pass every retrieved chunk to the model. Deduplicate URLs, rank by relevance and recency, cap text per source, and enforce a total evidence budget before extraction. The evidenceBudget function above is a small version of that guardrail.

Step 4: extract competitive signals with citations

The lightweight mode should not ask a model to invent a thesis. It turns retrieved evidence into cited signal candidates with deterministic rules, then leaves long-form synthesis to Deep Research.

function classifyTheme(row: Evidence): CompetitiveSignal["theme"] {
  const text = `${row.title} ${row.text}`.toLowerCase();
  if (/price|pricing|subscription|discount|margin/.test(text)) return "pricing";
  if (/customer|enterprise|segment|vertical|adoption/.test(text)) return "customers";
  if (/partner|alliance|collaboration|supplier/.test(text)) return "partnership";
  if (/revenue|guidance|margin|valuation|market share|stock/.test(text)) return "financial";
  if (/risk|regulat|supply|dependence|litigation|competition/.test(text)) return "risk";
  if (/roadmap|launch|product|platform|chip|service|feature/.test(text)) return "product";
  return "strategy";
}

function signalClaim(row: Evidence) {
  const cleaned = row.text.replace(/\s+/g, " ").trim();
  const excerpt = cleaned.length > 220 ? `${cleaned.slice(0, 217)}...` : cleaned;
  return `${row.title}: ${excerpt}`;
}

function extractSignals(target: string, peers: string[], focus: string, evidence: Evidence[]): Brief {
  const signals: CompetitiveSignal[] = evidence.slice(0, 12).map(row => ({
    company: row.company,
    theme: classifyTheme(row),
    claim: signalClaim(row),
    whyItMatters: `This ${row.kind} source may affect the ${focus} comparison for ${row.company}.`,
    sourceIds: [row.id],
    confidence: row.kind === "filing" ? "high" : row.kind === "market" ? "medium" : "low",
  }));

  return {
    target,
    peers,
    summary: `Retrieved ${evidence.length} source rows for ${companyList(target, peers).join(", ")}. Review the cited signal candidates below before escalating to Deep Research.`,
    signals,
    nextQuestions: [
      `Which ${target} claims are supported by filings rather than news?`,
      `Which peer changed positioning around ${focus} most recently?`,
      "Which signals need a full Deep Research report before sharing?",
    ],
  };
}

Good competitive intelligence is not just a summary. It separates:

What happened: a product launch, partnership, pricing change, acquisition, or risk disclosure
Where it came from: filing, news, market data, or company page
Why it matters: which part of the target's strategy it could affect
How confident the signal is: filings are stronger evidence than one-off web mentions

Step 5: check citations before writing the brief

A competitive intelligence agent should fail closed. If the model cites a source ID that does not exist, the code should reject the brief.

function verifyBrief(brief: Brief, evidence: Evidence[], expectedTarget: string, expectedPeers: string[]) {
  if (brief.target !== expectedTarget) throw new Error(`Unexpected target: ${brief.target}`);

  const validIds = new Set(evidence.map(row => row.id));
  const knownCompanies = new Set(companyList(expectedTarget, expectedPeers));

  for (const signal of brief.signals) {
    if (!knownCompanies.has(signal.company)) {
      throw new Error(`Unknown company in signal: ${signal.company}`);
    }
    for (const id of signal.sourceIds) {
      if (!validIds.has(id)) throw new Error(`Signal cites missing source ID ${id}`);
    }
  }
}

function cell(value: string) {
  return value.replace(/\|/g, "\\|").replace(/\n/g, " ").trim();
}

function markdownBrief(brief: Brief, evidence: Evidence[]) {
  const byId = new Map(evidence.map(row => [row.id, row]));
  const rows = brief.signals.map(signal => {
    const sources = signal.sourceIds.map(id => {
      const source = byId.get(id);
      return source ? `[${id}](${source.url})` : `[${id}]`;
    }).join(", ");
    return `| ${cell(signal.company)} | ${cell(signal.theme)} | ${cell(signal.claim)} | ${cell(signal.whyItMatters)} | ${signal.confidence} | ${sources} |`;
  }).join("\n");

  return `# Competitive intelligence brief: ${brief.target}

> Research only. This is not investment, legal, tax, or financial advice. It does not recommend buying, selling, or holding securities.

${brief.summary}

| Company | Theme | Claim | Why it matters | Confidence | Sources |
| --- | --- | --- | --- | --- | --- |
${rows}

## Next questions
${brief.nextQuestions.map(q => `- ${cell(q)}`).join("\n")}

## Source index
${evidence.map(row => `- [${row.id}] ${row.kind.toUpperCase()} · ${cell(row.company)} · [${cell(row.title)}](${row.url})`).join("\n")}
`;
}

This is the line between a useful research agent and a confident autocomplete wrapper: the brief is not allowed to outgrow its evidence packet.

Step 6: run the lightweight competitive intelligence agent

Add a small main function:

async function main() {
  if (!process.env.VALYU_API_KEY) throw new Error("Set VALYU_API_KEY in .env");
  const evidence = await collectEvidence(input.target, input.peers, input.focus);
  const brief = extractSignals(input.target, input.peers, input.focus, evidence);
  verifyBrief(brief, evidence, input.target, input.peers);

  const output = markdownBrief(brief, evidence);
  await writeFile("competitive-intel-brief.md", output);
  await writeFile("competitive-intel-evidence.json", JSON.stringify(evidence, null, 2));
  console.log(output);
}

main().catch(error => {
  console.error(error);
  process.exit(1);
});

Run it:

npx tsx --env-file=.env competitive-intel-agent.ts "NVIDIA" "AMD, Intel, Broadcom" "AI data center accelerators and inference infrastructure"

The output should be a cited signal-candidate brief, not a stock pitch. The source index is intentionally verbose because analysts need to inspect the underlying documents. Use the Deep Research mode below when you want a polished narrative report.

Add a Deep Research mode like the competitor analysis app

The lightweight agent above is best when you want fast, structured monitoring. But some competitor questions are too broad for a short retrieval pass:

“Map this competitor's product strategy across segments.”
“Compare the target's positioning against three public peers.”
“Find recent partnerships, product launches, and market shifts, then produce a report with citations.”
“Generate a PDF I can share with the team.”

That is where Deep Research fits.

The competitor-analysis app uses a two-panel flow:

user enters a competitor website and context
the backend creates a Valyu Deep Research task
the UI polls task status every few seconds
the UI polls a status endpoint and displays progress updates while research runs
the final result includes Markdown, citations, and a PDF export

The repo has the core configuration as:

model: fast for quick reports, with standard, heavy, and max available when deeper research and higher cost or latency are justified
asynchronous task creation
client-side polling to avoid server timeouts
output formats: Markdown and PDF
cited sources attached to the report

For this public-company agent, expose two modes:

Mode	Best for	Output
`brief`	recurring monitoring, dashboards, alerts	structured cited table
`deepresearch`	long-form competitor analysis, strategy memos, shareable reports	Markdown + PDF report

Here is the Deep Research task creator you can add to a backend route:

type DeepResearchTask = {
  deepresearch_id: string;
  status: "queued" | "running" | "completed" | "failed" | "cancelled" | "awaiting_input" | "paused";
};

async function createDeepResearchReport(target: string, peers: string[], focus: string): Promise<DeepResearchTask> {
  const response = await fetch("https://api.valyu.ai/v1/deepresearch/tasks", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Authorization": `Bearer ${process.env.VALYU_API_KEY}`,
    },
    body: JSON.stringify({
      input: `Create a competitive intelligence report for ${target}.

Compare against these public-company peers: ${peers.join(", ")}.
Focus area: ${focus}.

Include:
- company overview and market positioning
- product and segment comparison
- recent news and partnerships
- relevant SEC filing evidence
- market and financial context where available
- risks and open questions
- source citations for material claims

Do not give investment advice. Do not recommend buy, sell, or hold. Do not provide price targets, ratings, portfolio recommendations, or valuation judgments.`,
      model: "fast",
      urls: [],
      output_formats: ["markdown", "pdf"],
    }),
  });

  if (!response.ok) throw new Error(`Deep Research task failed: ${response.status}`);

  const data = await response.json() as DeepResearchTask;
  if (!data.deepresearch_id) throw new Error("Deep Research task did not return deepresearch_id");
  return data;
}

In a Next.js app, your UI should create the task, save the returned deepresearch_id, and poll a status route every 10 seconds until the report is complete. That architecture matters because deep research can run longer than typical serverless request limits.

A good Deep Research prompt is more specific than “analyze this company.” Give it:

target company and ticker
peer companies and tickers
focus area
date range
source preferences
output sections
explicit instruction to cite material claims
explicit instruction not to provide investment advice

The deep report is not a replacement for the lightweight agent. It is the escalation path when the question needs synthesis, source discovery, and a shareable report.

Implementation checklist

Before you ship this agent, check that you are preserving source IDs, separating source types, limiting evidence size, and routing broad research questions to Deep Research instead of forcing every job through the lightweight brief mode.

Common mistakes

1. Treating news as the whole story

News is fast, but filings are slower and often more precise. Use both.

2. Mixing companies without identifiers

Store ticker, CIK, exchange, and official website. Names alone are weak keys.

3. Letting the model invent a thesis

Ask for signals and follow-up questions. Do not ask for buy/sell recommendations.

4. Losing citations after summarization

Citation IDs should move through the whole pipeline: retrieval, extraction, brief, saved evidence.

5. Running Deep Research for every alert

Use the lightweight brief for recurring monitoring. Use Deep Research when a human needs a full report.

FAQ

What is a competitive intelligence agent?

A competitive intelligence agent is a workflow that retrieves information about competitors, extracts strategic signals, compares those signals across companies, and returns a cited brief or report.

Why use public-company sources?

Public companies publish regulated disclosures such as 10-K, 10-Q, and 8-K filings. Those filings can reveal strategy, risks, acquisitions, customer concentration, and segment trends that may not appear in ordinary web search results. Treat filings as primary disclosures, but still distinguish audited financial statements from unaudited quarterly data, forward-looking statements, and management commentary.

Can this agent use earnings calls?

Yes. Add an earnings-transcript retrieval step and label transcript evidence separately from filings and news. Management commentary is useful, but it is not the same as audited financial statements or SEC disclosures.

Is this investment advice?

No. This tutorial builds a research and monitoring tool. It does not recommend buying, selling, or holding securities.

When should I use Deep Research instead of search?

Use the lightweight Valyu Search workflow for fast, structured monitoring and source-linked signal candidates. Use Deep Research when you need a longer cited report, multi-source synthesis, PDF export, and progress tracking for a several-minute analysis task.