Building AI applications often requires more than connecting to an LLM. You also need reliable external data that can be combined with your own information.
SEC EDGAR is a valuable source for company research, financial analysis, and AI-powered research applications. But integrating external datasets into your workflows can require additional data ingestion, normalization, and maintenance work.
We've been exploring a simpler approach at Dataville.
Dataville provides access to curated external datasets through an API and MCP server, starting with sources including SEC EDGAR and Wikimedia.
Example: Building a sector research dataset
Imagine you're building an AI-powered research application that needs to compare companies within a particular industry.
You might want to combine:
- SEC filings: Public company disclosures and financial information.
- Proprietary research: Internal company notes, analyst observations, or other research.
- Structured analytics: A consistent way to join, filter, and analyze the resulting information.
One approach is to use DuckDB, an analytical database that makes it possible to query and combine datasets using SQL.
We've documented a workflow that uses Dataville to retrieve SEC filing data and DuckDB to combine it with proprietary research.
The result is a sector-specific research dataset that can support company comparisons, analysis, and downstream AI applications.
Follow the complete SEC EDGAR + DuckDB tutorial
Why external data matters for AI applications
As AI applications become more specialized, access to relevant external data becomes increasingly important.
Consider a few potential applications:
AI-powered financial research
Build applications that retrieve company information from SEC filings and combine it with internal research.
Competitive intelligence
Create structured comparisons of companies using external information alongside proprietary data.
Retrieval-augmented generation (RAG)
Use external reference material to provide additional context for AI-generated responses.
AI agents
Give agents access to external information through APIs or MCP integrations rather than relying exclusively on information embedded in the model.
These applications share a common requirement: getting useful external data into the application.
Reducing the work of external data integration
Traditionally, developers working with multiple external datasets may need to build separate integrations for each source.
That can involve retrieving data, transforming it into consistent formats, handling updates, and maintaining the resulting pipelines.
Dataville is designed to simplify this process by providing curated external datasets through a consistent API and MCP interface.
The initial sources include SEC EDGAR and Wikimedia, with plans to expand the available datasets.
For developers, the goal is straightforward: spend less time maintaining external data integrations and more time building applications.
Try the example
If you're working on AI applications that need external company information, we've published a hands-on walkthrough showing how to combine SEC filing data with your own research using DuckDB.
Follow the complete SEC EDGAR + DuckDB tutorial
You can also Explore the Dataville developer guides or try Dataville for free.
I'd love your feedback
We're early in building Dataville and interested in learning how other developers approach external data integration.
How are you managing external datasets in your AI applications?
Are you maintaining your own ingestion pipelines, using third-party APIs, or connecting external sources directly to AI agents?
And which external datasets would be most useful for the applications you're building?
I'd love to hear what's working, what's frustrating, and what you'd like to see improved.
We're still early with Dataville and would welcome feedback from developers building data-intensive AI applications.
How are you managing external datasets today? Are you maintaining your own pipelines, using third-party APIs, or connecting data directly to AI agents?
Top comments (1)
Great walkthrough, Kevin.
Pairing EDGAR filings with internal analyst notes in DuckDB is exactly the kind of setup we built Dataville for, and it’s nice to see it shown so clearly. The point about per-source pipelines rings true: getting the data is the easy part, while keeping it normalized and current is what eats the time. Full disclosure, I’m the CEO of Dataville, so I’m biased, but I’d love to hear how others here are handling external data for their agents today.
ai #database #duckdb