Production AI teams already track latency, token usage, error rates, and cost.
The next question is increasingly environmental:
How much energy did this inference use?
What carbon impact can be attributed to it?
Can water use be estimated?
Which values are measured, and which are modeled?
Can the result support an internal or external disclosure?
These are reasonable questions. They are also easy to answer badly.
A per-inference estimate can be useful without being a complete emissions inventory. The important part is preserving the boundary between operational visibility and organizational disclosure.
At CarbonLayer, we use a simple rule:
Show what was measured. Label what was derived. Qualify what was modeled.
What a per-inference metric actually describes
A per-inference metric describes a specific workload within a defined boundary.
That boundary might include:
The model or model class
Input and output size
Runtime or execution environment
Region or facility assumption
Time period
Energy-allocation method
Carbon-intensity source
Water-intensity source
Baseline used for comparison
This can provide valuable operational context. For example, an engineering team might compare two routing strategies by looking at latency, cost, energy, and modeled carbon attribution.
But that same value does not automatically describe the healthcare organization’s complete environmental impact.
Workload-level data It can help explain It cannot prove
Per-inference energy Relative workload efficiency A facility’s total electricity use
Carbon allocation Impact of a defined workload A complete Scope 1, 2, or 3 inventory
Water attribution A modeled comparison between workloads Facility water withdrawal or consumption
Savings estimate A modeled baseline comparison Measured environmental savings
The number can be valid within its boundary while still being insufficient for a broader disclosure.
Scope boundaries still matter
A per-request estimate should not be presented as a replacement for organizational accounting.
Scope 1 covers direct emissions from sources an organization owns or controls, such as fuel combustion or refrigerant releases.
Scope 2 covers purchased electricity. A workload-level carbon estimate may allocate emissions using an electricity-intensity coefficient, but that is not the same as a utility-meter reading or a full facility-level Scope 2 disclosure.
Scope 3 covers value-chain emissions, including vendor services, equipment, transportation, and upstream energy impacts. A model-routing comparison is not automatically a Scope 3 inventory.
The key distinction is simple:
A workload estimate can support a disclosure process without becoming the disclosure itself.
Carry the status with the number
Environmental data should not be separated from its provenance.
A dashboard that displays 0.04 kWh or 12.3 g CO₂e without explaining the source creates false confidence. The value may be calculated correctly, but the reader still needs to know what it represents.
We recommend using three explicit states.
Measured
The value comes from a directly sourced input with documented evidence, geography, period, and boundary.
Examples might include:
A facility meter reading
A provider-issued energy report
A documented regional intensity factor for a defined period
Derived
The value comes from a calculation, conversion, or aggregation.
Examples include:
Converting watt-hours to kilowatt-hours
Summing request-level records for a reporting period
Converting milliliters to liters
Aggregating regional values into a defined workload total
A derived value may inherit uncertainty from the inputs underneath it.
Modeled
The value is an allocation, estimate, proxy, forecast, baseline, or counterfactual comparison.
Examples include:
Allocating shared compute energy to an individual inference
Estimating water attribution from energy use
Comparing a carbon-aware route with a conventional baseline
Forecasting impact from expected request volume
A precise-looking modeled value is still modeled.
Why a measured coefficient can produce a modeled result
Consider a simplified carbon calculation:
allocated carbon = allocated energy × regional carbon intensity
The regional carbon-intensity value might come from a documented source. That source can be measured.
The allocated energy assigned to one inference may still depend on assumptions about shared infrastructure, workload utilization, model behavior, and scheduling.
Therefore, the resulting per-inference carbon value should usually be labeled modeled, even if one of its inputs is measured.
The calculation may be sound. The label still matters.
This distinction is especially important when values are copied into reports or presented to non-technical stakeholders. “Measured coefficient” and “measured workload impact” are not interchangeable claims.
Water attribution needs extra caution
Water is particularly easy to overstate.
A value such as waterMl may represent a modeled allocation based on energy use and a water-intensity coefficient. It is not automatically:
A facility meter reading
Total facility water consumption
Water withdrawal
Water discharge
A complete water inventory
A clinical or patient-related impact
The same applies to a comparison field such as savedWaterMl.
If the value represents the difference between a selected routing strategy and a baseline, it is a modeled comparison. It should not be presented as measured water savings unless the underlying measurement supports that claim.
For example:
savedWaterMl = baseline modeled water attribution
- selected route modeled water attribution
That can be useful for evaluating routing decisions. It does not mean a facility consumed that exact amount less water.
An illustrative inference record
The following is an illustrative data model, not a required API format:
{
"workload": {
"model": "example-model",
"region": "example-region",
"period": "2026-09-01/2026-09-30"
},
"performance": {
"latencyMs": 184,
"inputTokens": 920,
"outputTokens": 240
},
"impact": {
"energyWh": {
"value": 0.04,
"status": "modeled",
"method": "workload allocation"
},
"carbonGrams": {
"value": 0.012,
"status": "modeled",
"method": "energy allocation × regional intensity"
},
"waterMl": {
"value": 0.3,
"status": "modeled",
"method": "energy allocation × water-intensity coefficient"
}
},
"provenance": {
"sourceVersion": "example-source-version",
"geography": "example-region",
"assumptionsDocumented": true
}
}
The important design choice is not the field names. It is that the value and its qualification travel together.
Do not make engineers or reporting teams reconstruct the meaning of a number from a separate dashboard legend six months later.
Healthcare adds a privacy boundary
Environmental data is not automatically patient data.
However, healthcare operations can still be sensitive. Facility identity, location, timing, utilization, and workload patterns may reveal more than intended when combined.
Environmental telemetry should therefore avoid including:
Patient identifiers
Encounter IDs
Diagnoses
Treatments
Clinical notes
Claims about patient outcomes
A workload-level environmental record should describe the infrastructure and reporting boundary, not the patient.
Aggregation may also be appropriate. A monthly or regional report can reduce exposure from publishing individual workload timing or facility-level utilization patterns.
Privacy review and environmental disclosure review are separate processes. Both are necessary.
Aggregation does not remove uncertainty
Rolling up request-level values into a monthly total makes the number larger. It does not automatically make it more certain.
If the underlying records are modeled, the total remains based on modeled inputs.
If the underlying records use proxy data, the aggregate inherits that limitation.
If different regions or source versions were used, the aggregation should preserve those distinctions or explain how they were normalized.
A reporting-period total might be derived from modeled records. Those are two different descriptions, and both may be needed:
Derived describes how the total was calculated.
Modeled describes the nature of the underlying attribution.
Do not replace both with the simpler word “actual.”
A practical review checklist
Before including an inference-level value in an external report, confirm:
Boundary — Which workload, facility, provider, and reporting category does it cover?
Period — Is it historical, current, forecast, or counterfactual?
Source — Who supplied the coefficient, and which version was used?
Geography — Which region, grid, facility, or provider boundary applies?
Fallbacks — Was proxy, synthetic, degraded, or forecast data used?
Aggregation — How were individual records converted into the reported total?
Uncertainty — Which assumptions could materially change the result?
Review — Have engineering, sustainability, facilities, privacy, and reporting owners reviewed it?
This is not paperwork for its own sake. It is what makes an operational metric defensible.
A disclosure sentence engineers can stand behind
A useful disclosure should explain both the result and its limits.
For example:
During the reporting period, the healthcare AI workload processed [request count] requests. The reported carbon and water values are modeled workload allocations based on [method and source], not facility meter readings or a complete organizational emissions inventory.
That sentence is less dramatic than claiming a precise environmental saving.
It is also more useful.
The standard
AI infrastructure needs better environmental visibility. Production teams should be able to compare cost, latency, utilization, carbon, and modeled water without hiding the assumptions behind those values.
But better visibility does not mean presenting every estimate as a fact.
The standard should be:
Show what was measured.
Label what was derived.
Qualify what was modeled.
Preserve source, period, geography, and boundary.
Never imply what the system cannot establish.
For the full healthcare-specific framework, see the CarbonLayer healthcare emissions and water disclosure guide.
Top comments (0)