DEV Community: ches

Portable PL Test Equipment: Keeping Sample IDs Attached to the Data

ches — Thu, 15 Oct 2026 12:31:16 +0000

A folder with three files called reading_01.txt, reading_02.txt and reading_03.txt looks tidy enough.

Now someone asks which one came from sample PV-017, whether it was taken at the center or the edge, and which settings were active.

The filenames don't answer any of that.

That is the small software problem I want to look at here. When portable test equipment is used across several samples, the saved result needs an unambiguous link back to the sample and the measurement setup.

I work on technical content for TDM Technology. The hardware reference for this example is its SE10 portable PL metrology instrument, which lists local storage and a Bluetooth connection to a Windows PC.

That product description is not an API specification. It does not establish a file-export format or a public Bluetooth protocol. The example below starts with a completed file already on the computer; obtaining suitable files from the actual instrument software must be confirmed separately. There are no SE10 SDK calls here.

Start with the sample, not the filename

For this example, each local file gets a small JSON record beside it. This is often called a sidecar file. It holds the context without changing the original data.

I would begin with four references: the sample, the measurement position, the instrument and the setup.

PV-017 identifies the sample. P03 identifies a position on a photo or drawing kept with the session. portable-demo-01 identifies the instrument. setup-v3 points to a preserved settings record or notebook entry.

Those references need somewhere to point. A setup name is not much use if the settings behind it have been overwritten. Likewise, a position label needs a definition that another person can follow.

The code only records these labels. It cannot verify that the operator selected the right sample or used the stated settings. For a real workflow, I would make that confirmation part of the acquisition process, rather than reconstructing it from memory at the end of the day.

A small Python sidecar writer

This script reads a file, calculates its SHA-256 digest and writes the references into a neighboring JSON file. It leaves the input untouched and refuses to overwrite an existing sidecar.

Save it as pl_record.py. It uses the Python standard library and can be run with Python 3.10 or later.

"""Attach sample metadata to a completed local file; no device connection."""


import argparse
import hashlib
import json
from datetime import datetime, timezone
from pathlib import Path
from uuid import uuid4


def add_metadata(
    path: Path, *, sample: str, position: str, setup: str, instrument: str
) -> Path:
    refs = {
        "sample_id": sample.strip(),
        "position_ref": position.strip(),
        "setup_ref": setup.strip(),
        "instrument_ref": instrument.strip(),
    }
    if not all(refs.values()):
        raise ValueError("Sample, position, setup and instrument are required.")

    digest = hashlib.sha256()
    with path.open("rb") as source:
        for chunk in iter(lambda: source.read(1024 * 1024), b""):
            digest.update(chunk)

    record = {
        "schema_version": 1,
        "record_id": str(uuid4()),
        **refs,
        "logged_at_utc": datetime.now(timezone.utc).isoformat(),
        "file_name": path.name,
        "file_sha256": digest.hexdigest(),
    }
    destination = path.with_name(path.name + ".metadata.json")
    payload = json.dumps(record, indent=2, ensure_ascii=False) + "\n"
    with destination.open("x", encoding="utf-8") as output:
        output.write(payload)
    return destination


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("file", type=Path)
    parser.add_argument("--sample", required=True)
    parser.add_argument("--position", required=True)
    parser.add_argument("--setup", required=True)
    parser.add_argument("--instrument", required=True)
    args = parser.parse_args()
    try:
        print(add_metadata(
            args.file, sample=args.sample, position=args.position,
            setup=args.setup, instrument=args.instrument,
        ))
    except (OSError, ValueError) as exc:
        parser.exit(2, f"Error: {exc}\n")


if __name__ == "__main__":
    main()

To try it without an instrument, create a file named demo_reading.txt containing any clearly labeled synthetic text. Then run:

python pl_record.py demo_reading.txt --sample PV-017 --position P03 --setup setup-v3 --instrument portable-demo-01

The output is:

demo_reading.txt.metadata.json

There are now two files to keep together:

demo_reading.txt
demo_reading.txt.metadata.json

The script does not interpret the file as a spectrum or assume any particular PL data format. The .txt extension is just for this demonstration.

A missing file or blank reference produces an error. Running the same command again also produces an error, because Python's exclusive-create mode, "x", will not replace the existing metadata file.

That protects against an accidental overwrite by this script. It does not make the record immutable or protect it against someone editing it with another program.

Two fields that are easy to misread

The timestamp is called logged_at_utc, deliberately. It records when this script created the record, using a timezone-aware UTC datetime. It is not the acquisition time.

If the instrument supplies a trustworthy measurement timestamp, preserve it separately, together with the clock's timezone or reference. A file's modification time or the moment it arrives on a laptop should not silently become the time of measurement.

The second field is file_sha256. Python's hashlib computes the digest from the file's bytes. Saving it gives a later check something to compare against when looking for file changes.

It does not prove that a measurement is correct, identify a physical sample, or establish that two identical files came from the same acquisition. Two separate acquisitions might produce identical file contents. Do not automatically throw one away as a duplicate measurement just because its hash matches.

Each sidecar therefore gets its own record_id. Even that identifies this logging record, not a device-generated acquisition. A future instrument adapter would need to preserve the device's acquisition identity separately, if available.

Keep the first version small—and be clear about its limits

Run this only on completed files that are no longer being written. The script does not detect transfer completion, lock the input or recover a half-written sidecar after an interrupted write. It is a local example, not a multi-user lab database.

The next useful step would be a simple operator screen: select the sample and position, confirm the setup, attach the completed file, and review the record before moving to the next sample. The screen should make the active sample obvious. Convenience is not worth quietly carrying the previous sample ID into the next measurement.

For portable PL test equipment, that handover between the physical sample and the software record deserves attention. Wireless connectivity can be part of the hardware workflow; it does not tell an application what an operator is measuring.

The acceptance test I have in mind is modest. Hand the session folder, its setup records and its position drawing to a colleague. Can they work out which file belongs to which sample without asking the person who collected it?

That would be a useful first result, before writing any plotting code.

Thin-Film Measurement Data: What to Check Before Building a Dashboard

ches — Thu, 10 Sep 2026 07:38:38 +0000

Imagine a coating-line dashboard showing 1,200 nm for thirty seconds.

The film might be consistent. Or the application might have stopped receiving measurements and kept the last value on screen. Looking at the number alone, you cannot tell.

That is the software problem I want to focus on here: what needs to travel with a thin-film measurement so that an application can use it sensibly?

I work on technical content for TDM Technology. Its thin-film thickness guide covers measurement conditions for semiconductor, photovoltaic and display applications, including the distinction between wet-film, physical and model-derived thickness. Those distinctions also belong in the data model, not just in a PDF beside the instrument.

The JSON and Python below are a proposed integration example using synthetic data. They are not TDM SDK calls, a published device payload or a report from a production deployment.

Give the number a record

For this example, the record looks like this:

{
  "schema_version": 1,
  "device_id": "sensor-demo-01",
  "session_id": "session-demo-01",
  "sequence": 42,
  "run_id": "coating-demo-01",
  "recipe_id": "wet-film-demo-v1",
  "measured_at": "2026-09-10T08:00:00.000Z",
  "received_at": "2026-09-10T08:00:00.120Z",
  "film_state": "wet",
  "measurement_status": "valid",
  "thickness": {
    "value": 1.2,
    "unit": "um"
  }
}

The thickness is 1.2 micrometres, or 1,200 nanometres. The unit stays beside the original value, even if the application later standardizes everything to nanometres.

The other fields are there to answer fairly ordinary questions. Which device sent this? Which coating run was active? Was this a new observation or a replay? Which measurement recipe was in use?

Here, recipe_id refers to the measurement configuration, not the coater's operating recipe. The configuration behind it needs a version; reusing the same name after changing the settings would make old results ambiguous.

The device session and sequence number give this example a way to identify repeat deliveries. That depends on the adapter preserving identity across retries. Assigning a new ID every time the collector receives the same reading would defeat the point.

None of these field names should be assumed to exist in an instrument's output. Some may come from the device, some from the collector, and others from the production system. That ownership needs to be agreed before the integration is built.

Missing data should not become zero

There are at least three different situations worth separating:

A valid reading that meets the process specification.
A valid reading that falls outside the process specification.
No usable reading at all.

The second is potentially the most important result on the screen. It should not be discarded as bad data just because it missed the target.

The third should not become a zero or quietly inherit the previous value.

Here is a small Python function for the unit-conversion part. It only converts readings explicitly marked valid in the proposed schema. Unknown or unsuccessful measurement states return None. Malformed readings that claim to be valid raise an error.

This runs with Python 3.10 or later and needs no external packages.

from math import isfinite

TO_NM = {"nm": 1.0, "um": 1000.0, "µm": 1000.0, "μm": 1000.0}


def thickness_nm(record: dict[str, object]) -> float | None:
    """Return a normalized value only for an explicitly valid reading."""
    if record.get("measurement_status") != "valid":
        return None

    reading = record.get("thickness")
    if not isinstance(reading, dict):
        raise ValueError("Missing thickness object")

    value, unit = reading.get("value"), reading.get("unit")
    if isinstance(value, bool) or not isinstance(value, (int, float)):
        raise ValueError("Thickness must be a number")
    if not isinstance(unit, str) or unit not in TO_NM:
        raise ValueError(f"Unsupported thickness unit: {unit!r}")

    try:
        normalized = float(value) * TO_NM[unit]
    except OverflowError as exc:
        raise ValueError("Thickness is too large to represent") from exc
    if not isfinite(normalized) or normalized < 0:
        raise ValueError("Thickness must be finite and non-negative")
    return normalized


if __name__ == "__main__":
    samples = [
        {"measurement_status": "valid", "thickness": {"value": 1.2, "unit": "um"}},
        {"measurement_status": "no_signal", "thickness": None},
        {"measurement_status": "valid", "thickness": {"value": 1200, "unit": "nm"}},
    ]
    for sample in samples:
        print(thickness_nm(sample))

The output is:

1200.0
None
1200.0

The two valid readings normalize to the same value. The missing reading remains missing. I have also allowed both Unicode versions of the micro symbol, µ and μ, alongside ASCII um.

The isfinite check rejects NaN and infinity. Numeric strings and booleans are rejected too; this example expects a JSON number rather than trying to guess what the sender intended.

This is deliberately not a complete validator. It does not check timestamps, sampling positions or device calibration. Nor does it decide whether 1,200 nm is acceptable for a particular film. That requires a separate, approved specification.

The collector should retain the original record and its status or validation error. None is a value for the normalized output, not a reason to lose the explanation. A missing result should appear as a gap or an explicit unavailable state, rather than a fresh-looking measurement.

There are two times to keep track of

measured_at and received_at answer different questions.

The first belongs to the observation. The second records when the receiving application got it. In the synthetic record above, they are 120 milliseconds apart.

That difference is only meaningful as an age or delay estimate when the clock sources are understood and sufficiently synchronized. Local processing time may also sit between acquisition and delivery, so I would not automatically label it “network latency.”

If the device does not expose an acquisition timestamp, keep it unknown. The collector can still record arrival time, but it should not pretend that arrival time is when the film was measured.

Freshness also needs its own rule. A reading can have been valid when acquired and still be too old to describe the current process. For a first dashboard, I would show the time of the last received record and a stale-data indicator, with the threshold agreed for that process.

A mapping application also needs measurement coordinates and their reference frame. A sequence of timestamps is not a wafer map, and a single measurement location does not establish uniformity across a whole sheet.

Keep an untouched path through the system

The first version of this integration could be fairly small:

Device adapter
    -> raw record log
    -> validation and unit conversion
    -> measurement store
    -> read-only dashboard

The raw log comes before conversion so that a rejected record is still available for investigation. The adapter's job is to translate the documented device interface; it should not silently invent units or a successful status when those are missing.

Testing can start with synthetic records, before attaching hardware. The useful cases are not just a neat stream of valid values. Try an unsupported unit, a missing result, a duplicate delivery, a delayed record and a device restart. Check what is stored and what the dashboard actually shows in each case.

If averaging is added later, keep the original readings and identify the averaging window on the display. Mixing records from different runs or measurement recipes into one average would make a tidy chart harder to interpret.

Machine control is outside the scope of this version. Displaying measurements and deciding how a coater should respond are different responsibilities. A read-only path lets the team check units, timing and interpretation without also changing the process.

Once those checks work, there is a much better basis for adding charts, alerts or a connection to a manufacturing execution system. Until then, I would rather have a plain table with a trustworthy status and timestamp than a polished trend line that keeps moving when the data has stopped.