DEV Community

Cover image for Import Wikipedia tables without losing the revision behind the numbers
Tim Zinin
Tim Zinin

Posted on

Import Wikipedia tables without losing the revision behind the numbers

A spreadsheet copied from an encyclopedia article is difficult to audit after the article changes. The values may still look reasonable, but the importer has lost the page revision and the distinction between prose, ordinary data tables and layout elements. A reproducible import should retain the source article and revision beside the extracted rows.

The Wikipedia article and table extractor produces one article envelope containing its selected prose and ordinary wikitables. This is useful for a bounded research import where a person has already inspected the source article. It does not turn every visual arrangement on a page into a dataset.

Choose a source that fits the contract

The installed prefill points to the English Wikipedia article about permanent members of the United Nations Security Council. The README describes two historically accepted tables from that article. Their historical row counts are evidence about that accepted snapshot, not an assertion about today's revision.

{
  "articleUrls": [
    "https://en.wikipedia.org/wiki/Permanent_members_of_the_United_Nations_Security_Council"
  ],
  "includeText": true,
  "includeTables": true,
  "maxTables": 10,
  "maxRowsPerTable": 100
}
Enter fullscreen mode Exit fullscreen mode

The Actor accepts one through ten English Wikipedia article URLs. Table selection uses wikitable elements rather than infoboxes, navigation or layout tables. This matters when someone requests all the tables: the word all must be interpreted within the documented extraction scope.

Preserve the article envelope

Keep title, source URL, revisionId and the license link with the extracted tables. If you flatten nested rows into a warehouse table, copy the article identity and table position into each exported row. Otherwise two tables with similar column names can become indistinguishable after a union.

Treat normalized headers as importer metadata, not necessarily as stable identifiers across revisions. A renamed column can break a downstream join even if the underlying topic has not changed. Review structural changes before replacing a previously accepted table.

Bound the requested work honestly

maxTables can select at most twenty tables, and maxRowsPerTable can select at most two hundred data rows. The runtime also restricts the product of article count, table cap and row cap to ten thousand. Raising one bound can therefore require lowering another.

When a selected article exceeds the extraction caps, its partial notice must not be promoted into the complete-article table. Increasing a limit and retrying is a new source check against a potentially different revision. Retain the original partial record so the reason for the retry remains visible.

Separate completeness from suitability

A complete extraction means the selected unit fit the documented bounds. It does not mean that the article is an authoritative source for every decision, or that its statements have been independently verified. Retain references for human review where the use case depends on them.

Wikipedia content also comes with reuse obligations. Carry the returned attribution and license evidence into your publication workflow and consult the applicable source terms. A successful extraction is not a substitute for checking the intended downstream use.

Make a revision-aware import gate

Before accepting a new snapshot, compare its revision identity, table count and headers with the prior accepted import. Inspect added or removed tables before deleting warehouse data. Keep a separate record for a missing page and for a capped article. Neither should look like a complete table containing zero facts.

The resulting dataset is easier to maintain because every imported value has an article context. When a reviewer asks where a number came from, the answer includes the source and revision rather than only the date somebody last opened a spreadsheet.

Cost and verification scope

The current FREE-tier result price is $0.002 for the documented unit: One Wikipedia article with all its tables extracted as structured rows. The listing also has a $0.003 start event, charged per GB of Actor memory with a minimum of one event. A free diagnostic row does not refund that start event. Discounted platform tiers have their own rates; inspect your applicable tier before estimating a batch. These are listing prices read on 21 October 2026, not a receipt or a claim about customer spending.

The contract for this draft was checked against installed build 0.1.4 (UPcHrUYn8dbSXodzo). The input above is an illustrative schema example, not a newly executed run. Current source responses and useful output still need inspection when you choose to run it. No new cloud build or run was made to prepare this article.

Open the Actor and inspect its input and pricing.

AI-assisted draft; publication requires independent technical review.

Top comments (0)