Instrumenting a Super App Pilot: Measure Delivery Friction Before You Scale

A super app pilot can prove that a mini app loads inside a host application. That demonstration is useful, but it leaves the investment question largely unanswered.

The next decision usually concerns repeatability. Can the organisation deliver the second and third services with less friction? Can it update a service without another native release? Does reuse reduce integration work without moving risk into production? Does the customer journey remain reliable?

Answering those questions requires instrumentation before the pilot starts. Otherwise, the team finishes with a working service and a collection of impressions: development felt faster, integration seemed easier, and users appeared to stay in the app. Those observations are difficult to compare with the current delivery route.

This article describes a small measurement design for a TypeScript-based mini-app pilot. It uses OpenTelemetry for operational metrics and keeps delivery, engineering, and product evidence in their appropriate systems.

Begin with a metric contract

Write down the decision and the comparison before choosing instruments.

For example:

We will compare one mini-app service with a similar service delivered through the existing native or web route. Expansion requires an improvement in release lead time and recovery time, with no material deterioration in journey completion or reliability.

That statement identifies four measures, two guardrails, and a baseline. It also avoids claiming that one service can validate an ecosystem.

A basic metric contract should record:

Field	Example
Metric	Release lead time
Definition	Time from approved scope to successful production deployment
Unit	Seconds
Source	Delivery pipeline
Comparison	Median and p75 for comparable native releases
Segment	Release channel, publisher type, risk tier
Owner	Platform delivery lead
Decision use	Expansion gate

Definitions matter more than the number of metrics. If the native baseline begins when development starts but the pilot begins when a request is approved, the comparison is already distorted.

Keep three evidence planes separate

No single telemetry library can measure the whole business case well.

Delivery evidence comes from work-management and CI/CD systems: approval time, first commit, build completion, production deployment, rollback, and restoration.

Runtime evidence comes from the host app, mini app, gateway, and platform services: starts, completions, errors, duration, host-bridge calls, and recovery.

Engineering evidence comes from delivery records and team estimates: effort by discipline, integration days, security-review effort, and duplicated work. Runtime metrics should not be used as a substitute for these costs.

The scorecard can join these planes at the service and release level. Keep the raw data in its system of record.

Define low-cardinality attributes

OpenTelemetry metrics are well suited to counters and distributions, but attribute design needs restraint. User IDs, session IDs, raw URLs, order numbers, and unrestricted partner names can create an unbounded number of time series.

Use a small vocabulary that maps to a decision:

import type { Attributes } from "@opentelemetry/api";

type ServiceType = "campaign" | "loyalty" | "calculator" | "onboarding";
type PublisherType = "first_party" | "partner";
type ReleaseChannel = "mini_app" | "native" | "web";
type RiskTier = "low" | "medium" | "high";

export interface PilotAttributes extends Attributes {
  service_type: ServiceType;
  publisher_type: PublisherType;
  release_channel: ReleaseChannel;
  risk_tier: RiskTier;
}

Avoid adding a new attribute because it may be interesting later. Add it when a named comparison or operating action requires it.

Create the instruments

The example below uses @opentelemetry/api. It assumes that an SDK and exporter are configured elsewhere in the host or service. The API alone does not send data.

import { metrics } from "@opentelemetry/api";
import type { PilotAttributes } from "./pilot-attributes";

const meter = metrics.getMeter("super-app-pilot", "1.0.0");

const releaseLeadTime = meter.createHistogram(
  "pilot.release.lead_time",
  {
    description: "Seconds from approved scope to production deployment",
    unit: "s",
  }
);

const journeyStarted = meter.createCounter("pilot.journey.started", {
  description: "Number of pilot journeys started",
  unit: "{journey}",
});

const journeyCompleted = meter.createCounter("pilot.journey.completed", {
  description: "Number of pilot journeys completed",
  unit: "{journey}",
});

const journeyDuration = meter.createHistogram("pilot.journey.duration", {
  description: "End-to-end duration of a pilot journey",
  unit: "s",
});

const hostBridgeCalls = meter.createCounter("pilot.host_bridge.calls", {
  description: "Calls from a mini app to a governed host capability",
  unit: "{call}",
});

const hostBridgeFailures = meter.createCounter(
  "pilot.host_bridge.failures",
  {
    description: "Failed calls to a governed host capability",
    unit: "{call}",
  }
);

const rollbackRecovery = meter.createHistogram(
  "pilot.rollback.recovery_time",
  {
    description: "Seconds from rollback decision to verified service recovery",
    unit: "s",
  }
);

export function recordRelease(
  approvedAt: Date,
  deployedAt: Date,
  attributes: PilotAttributes
) {
  const seconds = (deployedAt.getTime() - approvedAt.getTime()) / 1_000;
  if (seconds >= 0) releaseLeadTime.record(seconds, attributes);
}

Recording release lead time from a deployment hook is usually more reliable than asking the mini app to report it. The pipeline knows whether deployment succeeded and can attach the approved timestamp from the work item.

Instrument a journey without identifying the customer

The pilot needs a stable journey name, but the metric stream does not need to identify the person completing it.

type JourneyOutcome = "completed" | "cancelled" | "failed";

export function startJourney(attributes: PilotAttributes) {
  const startedAt = performance.now();
  journeyStarted.add(1, attributes);

  return (outcome: JourneyOutcome) => {
    const durationSeconds = (performance.now() - startedAt) / 1_000;

    journeyDuration.record(durationSeconds, {
      ...attributes,
      outcome,
    });

    if (outcome === "completed") {
      journeyCompleted.add(1, attributes);
    }
  };
}

The completion rate should be calculated in the metrics backend from completed and started counters over the same interval and attribute set. Recording a client-side percentage produces values that are hard to aggregate correctly.

outcome is bounded to three values, so it is safe as an attribute. A raw error message would not be. If the team needs failure analysis, use a controlled error category such as validation, network, host_bridge, or downstream_service, and send detailed diagnostics to logs or traces.

Measure host capability reuse

Reusing identity, payments, location, messaging, or secure storage is often part of the platform hypothesis. Counting bridge calls alone does not prove economic reuse, although it can show whether the mini app uses the governed path and whether that path is reliable.

type HostCapability = "identity" | "payment" | "location" | "messaging";

export async function callHostCapability<T>(
  capability: HostCapability,
  attributes: PilotAttributes,
  operation: () => Promise<T>
): Promise<T> {
  const metricAttributes = { ...attributes, capability };
  hostBridgeCalls.add(1, metricAttributes);

  try {
    return await operation();
  } catch (error) {
    hostBridgeFailures.add(1, metricAttributes);
    throw error;
  }
}

Pair this runtime evidence with an engineering record. For each capability, document whether the pilot reused an existing host implementation, introduced an adapter, or created a new service. The business case can then distinguish genuine reuse from work shifted into the platform team.

Record recovery from the operator’s point of view

A mini app may be removable without submitting a new native binary. The useful measure is the full recovery interval: from the decision to withdraw or roll back until monitoring confirms that customers are no longer exposed to the failed version.

export function recordRollbackRecovery(
  rollbackDecidedAt: Date,
  recoveryVerifiedAt: Date,
  attributes: PilotAttributes
) {
  const seconds =
    (recoveryVerifiedAt.getTime() - rollbackDecidedAt.getTime()) / 1_000;

  if (seconds >= 0) rollbackRecovery.record(seconds, attributes);
}

Both timestamps should come from the control plane or incident workflow. A developer clicking “rollback” is an intermediate event, not verified recovery.

Compare distributions, not showcase releases

One fast deployment is weak evidence. Report the median and at least one upper percentile for repeated changes during the pilot. The median describes the typical path; p75 or p95 can reveal approvals, integration failures, or release contention hidden by the average.

The baseline needs comparable work. Match services by risk, scope, team maturity, and integration complexity where possible. If the sample is small, publish the individual observations alongside the summary. A precise percentage based on two releases can create more confidence than the data deserves.

A scorecard might include:

Measure	Primary source	Decision signal
Approval-to-production lead time	Work item + CI/CD	Change in delivery friction
Engineering effort per service	Delivery records	Reuse across services
Host-bridge failure rate	Runtime metrics	Safety of shared capabilities
Journey completion	Product events	Customer outcome guardrail
Rollback recovery time	Control plane + incident record	Operational containment
Support incidents	Service desk	Hidden operating cost

Add promotion gates before the result is known

The team should agree on gates while it can still be neutral about the outcome. For example:

• release lead time improves against the comparable baseline;
• recovery remains within the agreed operational target;
• bridge reliability stays above the service-level objective;
• journey completion remains within an accepted band;
• the second service demonstrates reuse without repeating the foundation work;
• no critical security or compliance issue remains open.

Use thresholds derived from the organisation’s own baseline and risk appetite. Generic targets copied from another platform will not account for local release controls, customer journeys, or service criticality.

Instrument the decision, then let the architecture earn its scope

A super app pilot does not need hundreds of metrics. It needs a short, defensible chain from an investment hypothesis to observable evidence.

Delivery systems show whether release friction changed. Engineering records show whether effort was avoided or moved. Runtime telemetry shows whether shared capabilities and customer journeys behaved safely. A stage gate brings those signals together without pretending that a first pilot has already produced a network effect.

Once the second and third services have been measured, the organisation can estimate marginal delivery cost with more confidence. Partner onboarding and ecosystem revenue can be added later, with their own evidence and operating costs. The pilot evidence can then determine the platform’s scope, replacing some of the uncertainty in the early forecast.

References

• OpenTelemetry metrics concepts
• OpenTelemetry Metrics API specification
• W3C Navigation Timing
• Apple Mini Apps Partner Program