DEV Community: Soumit S Rah

Your RAG Knows Today: Building News-Aware RAG with the Beans API

Soumit S Rah — Tue, 20 Oct 2026 12:34:27 +0000

Your users ask what happened today. Your RAG hands them last month's answer with immaculate citations. Congratulations: you built a very articulate time machine.

A company issues a recall this morning, posts a statement at lunch, and draws new coverage by evening. Your assistant should be able to find those documents when the question arrives. The model is good at writing the answer; it needs current material to write the right one.

Retrieval-augmented generation fixes the evidence problem by retrieving relevant material at request time and placing it in the model's context. For current public information, that material cannot be one frozen document collection assembled during a heroic Tuesday six months ago. It needs news, press releases, official statements, reports, legal records, research, and the coverage that appeared after the original announcement.

That is where the Beans API and Beans MCP fit: they provide the current publisher-material retrieval layer for a news-aware RAG system.

RAG gets more useful when time is allowed to pass

Traditional RAG examples are pleasantly stationary. Upload a handbook, split it into chunks, create embeddings, retrieve five passages, and ask the model which form Gerald in accounting should file.

Recent events are less considerate.

New Articles arrive after your last indexing job.
A press release presents one version of events; independent reporting presents several others.
A developing Story can contain coverage from multiple publishers across hours or days.
The newest Article is not necessarily the Article attracting the most attention.
A body may be unavailable even when the title, summary, source, URL, labels, and publication time are useful.

For this kind of application, retrieval is not merely “find a similar paragraph.” It is “find the relevant material, establish when it was published, distinguish the document type, inspect related coverage, and preserve the original URLs.” Tiny difference. Only most of the system.

Beans exposes publisher Articles, Sources, Stories, and external mention observations through one API. Article search supports a natural-language relevance query plus filters for content type, source, domain, author, category, region, entity, sentiment, tags, language, and publication dates. Results include stable IDs that can be used for selective detail and follow-up calls.

The evidence your application may actually need

Beans currently supports 14 request-filterable content types. The useful point is not the number; numbers look excellent in launch decks and are less helpful during an incident. The useful point is that your retriever can ask for the kind of evidence the question requires.

Concrete inputs include:

news and blog for publisher reporting and commentary;
press_release and official_statement for first-party corporate or institutional claims;
earnings_report and financial_report for company-performance updates;
lawsuit, enforcement_action, and contract for legal and regulatory monitoring;
research_paper, technical_documentation, and whitepaper for research and engineering context;
podcast and site for additional publication discovery.

Beans uses a changing catalog of public-information sources, so coverage and timing vary. That is worth saying plainly. “Everything on the internet, instantly” is not a product requirement; it is a cry for help.

Where recent-event RAG earns its keep

1. Incident and crisis assistants

Suppose a user asks, “What is known about the outage, and has the vendor responded?” The concrete material to look for might be a vendor newsroom statement, an engineering blog update, and reporting that checks the vendor's account.

A useful retrieval plan can combine:

recent news coverage for independent reporting;
an official_statement or press_release for the vendor's position;
technical_documentation for any changed operational guidance;
related Articles and Story membership to see whether multiple publishers are covering the same event;
mention observations to identify available external discussion around a selected Article URL.

The generated answer can then separate publisher reporting from the vendor's own statement and link to both. This is more useful than asking the model to remember an outage that began 14 minutes ago. It was busy being a matrix multiplication at the time.

2. Competitive and product intelligence

A product team may ask, “What did our competitor launch, how are publishers describing it, and is anyone paying attention?”

Retrieve press_release, official_statement, blog, and news Articles around the company and product. Use related-Article calls for adjacent reading, Story Articles for same-event coverage, and trending results or available trend observations for attention context.

This gives the model first-party claims, outside coverage, and the shape of the conversation. It also reduces the chance that your competitive brief is simply a press release wearing a fake mustache.

3. Earnings and company-monitoring copilots

For questions such as “What changed this quarter, and how was it reported?”, look for an investor-relations earnings release and a quarterly financial filing, then retrieve relevant press releases, official statements, and news coverage from the available Beans records. Start with earnings_report and financial_report to keep those documents distinct from the reporting about them.

The first-party documents supply the company's published account. Publisher Articles add interpretation and context. Date filters keep the retrieval window tied to the reporting period instead of inviting a beloved result from 2023 to wander into the prompt.

4. Legal and regulatory watch

Compliance, policy, and risk tools often need to track a company, industry, or jurisdiction across enforcement_action, lawsuit, contract, official_statement, and news material. Think of an agency enforcement notice, a court complaint, the company's response, and subsequent reporting as separate pieces of the answer; search for whichever are available.

Beans entity, region, category, Source, and date filters make that retrieval explicit. Your application can preserve each Article's type and URL, then instruct the model to describe what each source says rather than blending allegations, enforcement notices, and corporate responses into one legally adventurous paragraph.

5. Research and engineering radar

Developer tools can monitor research_paper, technical_documentation, whitepaper, blog, and news material for a technology or organization.

That supports questions like:

What new research appeared this week?
Did the project publish technical guidance after the announcement?
Which publisher Articles discuss the paper rather than merely repeat its title with impressive confidence?
Is this one Article, several related Articles, or a broader Story?

The model still needs to evaluate evidence and communicate uncertainty. Beans supplies retrievable material and structured context; it does not turn a whitepaper into peer review by the healing power of JSON.

A practical REST retrieval flow

The Beans route guide recommends small, intentional calls. For RAG, a sensible flow is:

Search a compact candidate set with GET /beans/articles/search.
Filter by document type, date, entity, region, Source, or other known constraints.
Select candidates across document types so one busy news cycle does not crowd out the original statement.
Request GET /beans/articles/{id}?full_content=true only for the Articles chosen for context.
Follow Story, similar-Article, Source, or mention links only when the question needs them.
Build the model context with titles, publication times, content types, summaries or available bodies, and canonical URLs.
Require the answer to cite those URLs and distinguish first-party material from publisher reporting.

Here is a small server-side JavaScript example. It searches the past seven UTC days across several evidence types because content_type accepts one value per request. Each Article's published_at tells you when its publisher says it was published; each collection's meta.as_of tells you when Beans produced that response.

const BASE_URL = "https://api.cafecito.tech";
const API_KEY = process.env.CAFECITO_API_KEY;

async function beansGet(path, query = {}) {
  const url = new URL(path, BASE_URL);
  for (const [key, value] of Object.entries(query)) {
    if (value !== undefined && value !== null && value !== "") {
      url.searchParams.set(key, String(value));
    }
  }

  const response = await fetch(url, {
    headers: { Authorization: `Bearer ${API_KEY}` },
  });
  if (!response.ok) throw new Error(`Beans returned HTTP ${response.status}`);
  return response.json();
}

export async function retrieveRecentContext(question) {
  const from = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000)
    .toISOString()
    .slice(0, 10);
  const contentTypes = ["news", "press_release", "official_statement"];

  const pages = await Promise.all(
    contentTypes.map((contentType) =>
      beansGet("/beans/articles/search", {
        q: question,
        content_type: contentType,
        from,
        limit: 8,
      }),
    ),
  );

  const firstByType = pages.flatMap((page) => page.data.slice(0, 1));
  const remaining = pages.flatMap((page) => page.data.slice(1));
  const selected = [
    ...new Map(
      [...firstByType, ...remaining].map((article) => [article.id, article]),
    ).values(),
  ].slice(0, 6);

  const details = await Promise.all(
    selected.map((article) =>
      beansGet(`/beans/articles/${article.id}`, { full_content: true }),
    ),
  );

  return {
    from,
    asOf: Object.fromEntries(
      pages.map((page, index) => [contentTypes[index], page.meta.as_of]),
    ),
    articles: details.map(({ data }) => ({
      title: data.title,
      publishedAt: data.published_at,
      contentType: data.content_type,
      source: data.source,
      text: data.content ?? data.summary ?? data.title,
      url: data.url,
      storyId: data.story_id,
    })),
  };
}

The first pass reserves a place for each content type that returns a match, then fills the remaining slots. A press release does not disappear merely because eight news Articles arrived first. Adjust the seven-day window to fit the question.

Beans collection responses include data, pagination, and meta. Continue pagination only by returning pagination.next_cursor unchanged on the same route with the same filters. Do not decode it. There is no hidden treasure inside, only a future support ticket.

Stories are useful when one event becomes 27 headlines

Article similarity and Story membership answer different questions.

GET /beans/articles/{id}/similar returns related publisher reading.
GET /beans/stories finds related-coverage groups.
GET /beans/stories/{id} returns Story detail.
GET /beans/stories/{id}/articles returns the Articles assigned to that Story.

For RAG, Story retrieval is useful when the answer should reflect how an event was covered across publishers rather than treating every headline as a separate event. It also gives your application a cleaner unit for deduplication and follow-up research.

Do not assume that every similar Article belongs to the same Story. Related is not identical. Software occasionally requires nouns to retain their meanings.

Or let the agent use Beans through MCP

If your application already has an MCP-capable agent, connect it to:

https://api.cafecito.tech/beans/mcp

The server uses the same Bearer API key as REST. Beans MCP exposes tools such as searchArticles, getLatestArticles, getTrendingArticles, getArticle, getSimilarArticles, getArticleMentions, listStories, getStory, listStoryArticles, and Source or taxonomy discovery tools.

The tool sequence is the same deliberate retrieval flow:

searchArticles or another collection tool finds candidates.
The agent reads real UUIDs from data.
getArticle retrieves only selected detail.
Story, similar, mention, or Source tools add context only when needed.
The agent cites the returned Article URLs.

MCP removes some integration plumbing. It does not remove the need for tool policy, call limits, source attribution, or a stern instruction against inventing UUIDs.

Production details that prevent exciting weekends

Keep the API key on the server or in the agent client's secret store.
Treat empty data: [] as a successful retrieval with no matches.
Use score_threshold only when q is present.
Preserve Article, Source, and Story UUIDs exactly.
Keep latest, top-headlines, trending, and relevance search conceptually separate.
Request full_content=true only for selected Articles and handle missing content.
Retain canonical Article URLs for attribution.
Treat categories, entities, sentiments, ideology, trend observations, and generated answers as context that may require verification—not fresh tablets from the mountain.

Most importantly, content availability is not a grant to republish, redistribute, archive, train on, or otherwise reuse it. Review the Third-Party Content Policy and assess the rights required for your application.

The short version

The next time someone asks what changed today, retrieve the dated material, compare the first-party account with publisher reporting, and answer with links to the original documents.

Beans gives developers REST and MCP access to searchable publisher Articles, Sources, Stories, related reading, and external mention observations. You decide what evidence the question requires, retrieve a compact candidate set, enrich only the selected records, and give the model dated, typed, attributable context.

Start with the Beans developer guide, use the workflow examples for call sequencing, or connect an agent through the Beans MCP guide.

Your model can keep doing the prose. Give it Beans for the part where today actually happened.

Launching Espresso API & MCP — Curated Business Intelligence for AI Agents & Dashboards

Soumit S Rah — Tue, 30 Jun 2026 12:01:00 +0000

It is my pleasure to introduce Project Cafecito’s next product — Espresso API & MCP.

The API is part of the larger Espresso product suite, which also includes Espresso Publications: Editorials, X, Threads — our curated digests and deeper human-readable analysis built on the same high-signal intelligence layer. Think of it as concentrated shots of structured intelligence: event digests, synthesized signals, tags, and relationships — served fresh for dashboards, monitoring workflows, and AI agents.

Like everything we ship these days, it comes with a full MCP server so your agents can sip directly without burning tokens on JSON parsing.

How Events and Signals Are Produced

Events are generated from a wide range of public data sources — product price changes, stock market movements, SEC filings, earnings reports, product announcements, corporate partnerships, and more.

Signals are the abstraction layer on top. They combine multiple similar events across an extended timeline to identify event chains, causal relationships, and commonality between various entities.

What Espresso actually gives you

Clean market event and signal records with timestamps, and domain-specific fields like actions, events, impact_level, marco_context, cross_domain_impacts etc.
Semantic search with adjustable accuracy.
Tag filtering.
Cross event relationships.
Token-optimized MCP-friendly response.

Quick Start + Python Examples

import requests
import os

API_KEY = os.environ["CAFECITO_API_KEY"]
BASE_URL = "https://api.cafecito.tech"

headers = {"Authorization": f"Bearer {API_KEY}"}

Semantic search for signals (plain text for MCP/agents)

resp = requests.get(
    f"{BASE_URL}/espresso/signals",
    headers=headers,
    params={
        "q": "market volatility and interest rate impact",
        "acc": 0.75,
        "limit": 5,
        "response_type": "text"
    },
    timeout=30
)
resp.raise_for_status()
print(resp.text)

Filter signals by tags

resp = requests.get(
    f"{BASE_URL}/espresso/signals",
    headers=headers,
    params={
        "tags": ["usa", "policy_reform"],
        "from": "2026-05-01",
        "limit": 10
    },
    timeout=30
)
resp.raise_for_status()
print(resp.json())
Combine semantic + tags

resp = requests.get(
    f"{BASE_URL}/espresso/signals",
    headers=headers,
    params={
        "q": "regulatory changes affecting tech",
        "tags": ["regulation","openai"],
        "acc": 0.8,
        "limit": 8
    }
)
resp.raise_for_status()
print(resp.json())

Sample Response Body (Events)

[
     {
        "actions": [
            "2026-06-28 Firefighters died and injuries occurred at Wyoming-Utah border.",
            "2026-06-28 Large wildfire destroyed parts of ski resorts."
        ],
        "briefing": "On June 28, 2026, three U.S. firefighters died while battling rapidly spreading wildfires near the Colorado-Utah border; approximately 100 sq km burned. Temperatures reached 34°C with strong winds, prompting mass evacuations. The Snyder Fire merged with others, causing significant damage to infrastructure like ski resorts. Causes include severe drought and human factors. This incident reflects escalating regional wildfire risks driven by climate change.",
        "companies": [
            "us_federal",
            "us_state"
        ],
        "cross_domain_impacts": [
            "public_safety: Increased risk of civilian casualties.",
            "tourism: Disruption of winter sports facilities.",
            "environmental: Habitat loss in mountainous areas."
        ],
        "event_type": "wildfire_outbreak",
        "future_outlook": "Continued extreme fire seasons expected without mitigation efforts.",
        "id": "091726f8-421a-566d-9db8-339625f2ed9e",
        "impact_level": "high",
        "macro_context": "western_us_climate_crisis",
        "regions": [
            "colorado",
            "utah"
        ],
        "reported": "2026-06-28T22:49:07Z",
        "tags": [
            "wildfire",
            "climate_change",
            "us",
            "emergency_response"
        ]
    },    
    {
        "actions": [
            "2026-06-28 Governments of United States and Iran agree to cease fire",
            "2026-06-30 Bilateral talks scheduled in Oruz"
        ],
        "briefing": "On June 28, 2026, U.S.-Iran governments reached an emergency agreement halting military clashes amid regional tensions. The pause prevents broader war while resuming negotiations by June 30 focused on Ormuz Strait security. This diplomatic step averts immediate market shocks but depends on both sides lowering aggressive rhetoric for tangible outcomes.",
        "cross_domain_impacts": [
            "energy_markets: Reduced oil volatility expected",
            "security_framework: Gulf stability improved temporarily"
        ],
        "event_type": "diplomatic_ceasefire",
        "future_outlook": "Talks may stabilize but risk re-escalation without sustained engagement.",
        "id": "6b4562d2-0a5c-540f-a39e-f1d5cad4ee5f",
        "impact_level": "high",
        "macro_context": "middle_east_conflict_resolution",
        "products": [
            "petróleo"
        ],
        "regions": [
            "gulf",
            "ormuz"
        ],
        "reported": "2026-06-28T21:52:00Z",
        "tags": [
            "diplomacy",
            "gulf",
            "iran",
            "us",
            "fuel"
        ]
    }
]

Full documentation is available in the Developer Portal.

Who is this for?

If you’re building AI agents, executive dashboards, automated briefings, or RAG pipelines that need real business/market/policy context — Espresso is the noise-free high-signal layer you’ve been missing.

Current status

v0.1 is live and open. It’s early — we’re still pruning bugs and adding more signals — but it’s already useful enough that I’m using it internally.

Try Espresso here | Get your API key & full docs

So… what now?

Fire it up and start sipping. It's free for a while dude!

Drop a comment below and tell me:

What signals or use cases would make this 10x more useful?
Any brutal feedback?
Or just share what you’re building. I read every single reply (yes, even the mean ones).

Happy building. May your signals stay strong and your agents well-informed. ☕

PS: This is part of the growing Cafecito stack (Beans + Espresso + more brewing). If you liked the no-BS vibe from the Beans launch, you’ll enjoy what’s coming next.

See you in the comments.

Espresso vs Beans — When to Use Which

Beans API is your bare metal news and blogs API. You query it when you want raw content being published by different publishers, authors, and news sites. Great for RAG, full article retrieval, or when you need the original source material. Oh by the way, we just added a new route for tracking content propagation across publishers and mentions on social media. It’s good 💩, check it out.

Espresso API is the structured, curated market intelligence layer. It takes multiple information sources, cross-correlates them, and gives you the data that actually matters for business decision-making — without all the pointless crying, whining, bitching, and moaning noise.
In short:

Beans = the full firehose.
Espresso = the refined shot you actually drink.

I Built Beans — A Semantic News & Blogs API & MCP for AI Agents and RAG

Soumit S Rah — Mon, 16 Mar 2026 17:41:43 +0000

☕ After a whole lot of brewing, we are excited to finally launch Beans API & MCP from Project Cafecito.

News APIs are over-priced. Most of them lack vector search capability. Almost all of them lack new propagation timeline and related news finding. Beans is different. We built it to beat market price and designed it for developers working on AI agents.

What is Beans?

It's an aggregation + search service that pulls fresh content from 7,000+ (We are adding new sources every week) news and blog publishers every day. No crawling, no maintenance — just clean, structured data.

Key Features

Natural language semantic search — Ask in plain text using vector-powered queries (with tunable accuracy)
Intelligent filtering — By category, named entities, geography, content type, source, time range, and more
Trend scoring — Surface what's actually gaining traction based on real social shares, likes and comments
Built-in AI enrichment — Automatic extraction of entities, sentiment, topics, and regions
Story propagation mapping — Automatically connect related articles across publishers to track how narratives spread
Clean, consistent JSON output perfect for pipelines

Whether you're building AI agents, RAG systems, market intelligence tools, automation workflows, or smart dashboards — Beans gives you high-quality, enriched news data without the usual headaches.

Current Status

Free Tier v0.1 is now live and open for you to play with. It's still early — expect the brew to get stronger with your feedback.

Create a free API key and get crackin'.

We would love to get your feedback, bug reports, and feature requests. Drop a comment below:

What use case would you use this for?
What features would make it 10x more useful for your workflow?

Happy building! 🚀