A vendor's built-in export button is designed to satisfy a support ticket, not to power a migration. If you actually need your data portable, the safer move is owning a small export script yourself, one that runs on a schedule and gives you a known-good copy independent of whatever the vendor's UI decides to include.
The Requests library is the natural starting point for this in Python, since most vendor APIs are plain REST over HTTPS and Requests handles the pagination and retry patterns below with minimal boilerplate.
Step 1: Identify the real source of truth
Before writing anything, map which objects actually matter. Not every table in a vendor's schema is worth exporting, but skipping the wrong one means discovering a gap mid-migration. Prioritize anything with relational structure (deals linked to contacts, tickets linked to accounts) since that structure is what most built-in export buttons flatten away.
Step 2: Pull via the documented API, not the UI export
Most SaaS platforms expose a REST API with pagination and rate limits. Use that instead of the UI export feature, because API responses typically preserve IDs and relationships that a CSV export drops.
import requests
import time
def export_all(endpoint, headers, page_size=100):
results = []
cursor = None
while True:
params = {"limit": page_size}
if cursor:
params["cursor"] = cursor
resp = requests.get(endpoint, headers=headers, params=params)
resp.raise_for_status()
data = resp.json()
results.extend(data["items"])
cursor = data.get("next_cursor")
if not cursor:
break
time.sleep(0.2) # respect rate limits
return results
Step 3: Preserve relationships, not just fields
Store foreign keys alongside the records they reference, even if the new system will eventually need different IDs. You can remap IDs during a real migration. You can't reconstruct a relationship that was never captured in the first place. If you're storing exports as plain files rather than a database, a format like JSON Lines keeps each record independently parseable, which makes partial recovery much easier if a later export run gets interrupted partway through.
def export_with_relationships(deals, contacts):
contact_lookup = {c["id"]: c for c in contacts}
enriched = []
for deal in deals:
contact = contact_lookup.get(deal.get("contact_id"))
enriched.append({
**deal,
"contact_email": contact["email"] if contact else None,
"contact_name": contact["name"] if contact else None,
})
return enriched
This kind of denormalization at export time costs almost nothing computationally and saves enormous effort later, since you won't need the original vendor's system alive just to resolve a foreign key.
Step 4: Run it on a schedule, before you need it
Set this script up as a weekly or monthly job, storing exports somewhere durable like object storage. The value isn't the export itself, it's having a recent, known-good copy on the day you actually decide to leave, instead of racing a support ticket during a contract deadline.
A simple cron entry or scheduled cloud function is enough for most teams. The frequency matters less than the consistency. A monthly export you can trust beats a "we'll figure it out when we need it" plan every time, because the day you actually need it is rarely a day when the vendor relationship is going well.
Step 5: Validate the export by reconstructing a record
Periodically pick one real record and try to fully reconstruct it from your export alone. If you can't, your export is missing something, and it's much cheaper to find that gap during a routine check than during an actual migration crunch.
Automate this validation where you can. A script that spot-checks a handful of exported records against the live system's API, comparing field-by-field, will catch schema drift long before a human reviewer would notice the export quietly stopped capturing a field the vendor added last quarter.
Step 6: Keep the schema documented alongside the export itself
The export script's own output format needs documentation too, or you're just relocating the same problem one layer down. A README next to the export bucket describing what each field means, which fields are foreign keys, and which fields changed meaning over time, turns a pile of JSON files into something an unfamiliar engineer could actually use during a real migration six months from now, possibly without you in the room.
Step 7: Handle pagination and rate limits like they're permanent, not incidental
It's tempting to write a quick script that assumes the API will always return everything in a single page, especially when your dataset is small today. Don't. Datasets grow, and a script that silently truncates results once you cross a pagination threshold is worse than no script at all, because it creates false confidence in an export that's actually incomplete.
def export_all_paginated(base_url, headers, page_size=100, max_retries=3):
all_items = []
cursor = None
while True:
for attempt in range(max_retries):
try:
resp = requests.get(
base_url,
headers=headers,
params={"limit": page_size, "cursor": cursor} if cursor else {"limit": page_size},
timeout=30,
)
resp.raise_for_status()
break
except requests.RequestException:
if attempt == max_retries - 1:
raise
time.sleep(2 ** attempt)
data = resp.json()
all_items.extend(data["items"])
cursor = data.get("next_cursor")
if not cursor:
return all_items
Building in retries and exponential backoff from the start means the script keeps working reliably as your data volume grows, rather than needing a rewrite the first time it hits a rate limit mid-export.
Step 8: Store more history than you think you need
Keep several months of prior exports, not just the latest one. If you discover a data quality issue, whether it's a vendor bug or something in your own script, having historical exports lets you pinpoint when the problem started rather than only knowing it exists in the current snapshot. Object storage is cheap enough that this is rarely a meaningful cost tradeoff against the diagnostic value it provides. A standard object store like Amazon S3 or an equivalent from another cloud provider is more than sufficient for this, and versioned buckets give you the historical retention almost for free.
Why this is worth doing even if you never migrate
Even teams with no near-term plans to switch vendors get real value from this pipeline. It becomes a natural audit trail, a disaster-recovery hedge if the vendor ever has a serious outage or data loss incident, and a source of truth you can query directly for analysis without hitting API rate limits against the live system. The migration-readiness benefit is real, but it's often the smallest of the three reasons teams end up glad they built this.
Step 9: Monitor the export job itself, not just the vendor
A scheduled export script that silently fails is arguably worse than no script at all, since it creates false confidence that a safety net exists when it actually stopped working months ago. Wire basic monitoring into the job: an alert if it fails to run, and a sanity check comparing the new export's record count against the previous run's count. A sudden, unexplained drop in record count is often the first sign something changed on the vendor's side, whether that's an API contract change, a permissions issue, or a genuine data loss event worth investigating immediately.
Step 10: Treat schema changes on the vendor's side as expected, not exceptional
Vendors change their APIs. Fields get renamed, deprecated, or restructured, usually with advance notice buried in a changelog nobody on your team reads regularly. Build your export script defensively, so a missing field logs a warning rather than crashing the entire job, and review those warnings periodically rather than only discovering a schema change when someone actually needs the missing data during a real migration.
Putting it together
None of these ten steps require exotic tooling or a dedicated team. A single well-monitored script, running on a schedule, storing structured exports with preserved relationships in durable storage, gets most teams most of the portability benefit that otherwise depends entirely on trusting a vendor's own export feature to be complete. The upfront investment is measured in hours, not weeks, and it pays for itself the first time a migration decision needs to happen faster than a vendor's support queue can accommodate.
This kind of proactive export discipline is exactly the countermeasure to the lock-in risk we break down in our framework for evaluating vendor lock-in before you sign a SaaS contract. Data portability is one of four dimensions worth scoring, and owning your own export pipeline is the practical way to keep that score in your favor regardless of what the vendor's contract says.
If you're building this kind of pipeline and want a second set of eyes on the schema design, that's the kind of data integration work we do often.
Top comments (0)