OSDU without a cloud: a LocalStack-style emulator for the OSDU data platform

The Open Subsurface Data Universe (OSDU) is a solid set of service contracts for
the energy industry: Storage, Search, Dataset, File, Workflow, Legal, Schema,
Entitlements. It powers real data platforms and real subsurface workflows.

But there is a gap between the contracts and the day-to-day reality of anyone
who writes code against them. A real deployment means Kubernetes, OpenSearch,
Redis, PostgreSQL, and an OIDC provider. That is reasonable for a platform team.
It is a heavy way to answer a much smaller question, which is: "does my client
actually work?"

This project answers that question without any of the platform. It is a
LocalStack-style emulator for OSDU: the same HTTP surface, everything in
memory, nothing else to install.

The idea

A single Python file, server/server.py, implements the OSDU service APIs.
A client built against a real deployment points at http://localhost:8089
instead and runs unchanged. Think LocalStack for OSDU:

• no cloud,
• no OpenSearch,
• no OIDC,
• no persistence layer,
• restart the process, and the platform is fresh again.

That makes it useful in the three places where a real deployment is annoying:
local development, tutorial walkthroughs, and CI.

Quickstart

docker run --rm -d --name osdu-well360 -p 8089:8089 ghcr.io/sohaibqasem/osdu-lite

That is the whole setup story. latest tracks main, every v* tag releases
its own image, and the images are multi-arch (amd64 and arm64).

From the source tree the same thing is make start, with make smoke to verify
health, entitlements, legal tags, kinds, and search.

What is emulated

Service	API	Highlights
Storage	v2	PUT/GET/PATCH/DELETE, version history, copy, batch ops, kinds, cursors
Search	v2	Lucene-style queries, cursor queries, aggregations, as-owner, health
Dataset	v1	registration, storage/retrieval instructions, registry, revoke, soft delete
File	v2	metadata, upload URLs, signed delivery, getLocation
Workflow	v1	deploy/list/run lifecycle, Osdu_ingest manifest ingestion
Legal	v1	tag CRUD, validate, batch retrieve, properties, query operators
Schema	v1	kind list, schema create/update/system
Entitlements	v1/v2	groups, members, membersCount, roles, appIds

Two behaviors matter once you write real code against it:

• Ingest is validated like a deployment. A record needs a known kind, acl.viewers/acl.owners, and legal.legaltags that exist in the Legal service. Failure returns 400 with a reason, so a client cannot silently ship invalid shapes.
• Storage and Search are asynchronous. A record is stored immediately but only appears in Search after the simulated indexer delay (0.7 s default). This reproduces the classic "I just ingested it, where is it?" moment on a real OSDU.

Dataset retrieval carries integrity data too: retrieval instructions include
size and SHA-256 for every known file, and the demo client aborts if the
downloaded LAS mismatches.

The reference example: Well 360

The repo is not just an API stub. It ships a complete Well 360 integration in
the shape of the official OSDU tutorial: Well -> Wellbore -> WellLog -> Dataset
-> LAS.

Three demo scripts show the progression:

./examples/well360/demo/run-tutorial.sh      # record-by-record ingest + LAS upload + Java client
./examples/well360/demo/run-full-sample.sh   # load all 3 Wells + 4 Wellbores + 4 WellLogs + 4 Datasets
./examples/well360/demo/run-manifest.sh      # register Dataset, then ingest a Manifest via Osdu_ingest

The star is a Java consumer (Well360App) compiled against the real
org.opengroup.osdu:os-core-common SDK. It searches well EAGLE-01, reads
record W-1001, resolves its Wellbores and WellLogs, fetches the Dataset
retrieval instructions, downloads and checksums the LAS, then runs a cursor
search. It never writes state, which makes it a safe read-only probe of the
whole chain.

Tests and CI

The test suite is zero-dependency stdlib unittest: no Docker, no live server,
make test works anywhere. 73 tests cover record validation and version
history, delete and purge, the search query language and cursor lifecycle, the
indexer delay, Dataset integrity, File v2, Workflow manifest ingestion, LegalTag
operators, Schema, entitlements RBAC, and TTL cleanup.

CI is deliberately offline and deterministic: the unit suite, a live smoke
test, and a client contract check that pins the os-core-common API surface the
demo consumes. No CI step depends on an external package registry.

Moving to a real OSDU

Everything runs against defaults that are easy to override: base URL,
partition, bearer token, ACL groups, and approved LegalTags. The Java
Search/Storage code stays the same against a real deployment; just confirm its
routes and schema versions before ingesting. Production hardening (OIDC, TLS,
retries, metrics) is intentionally out of scope for an emulator.

Why this exists

Some ideas are easiest to demo when they fit in one file and cost nothing to
spin up. OSDU-Lite exists so your tutorial, your client integration, or your
next talk can start with docker run and skip the part where you provision a
cloud account to try it.

Code, docs, and changelog: github.com/sohaibqasem/osdu-lite