DEV Community: TECHWORKSLAB

Writing a Pinnacle 21 style check in R, and why the check is not the evidence

TECHWORKSLAB — Sat, 31 Oct 2026 01:33:06 +0000

Conformance checking feels like the end of the job. You run the validator, the report comes back clean or close to it, and the dataset is pronounced ready. We want to make a narrower and more useful point: the check is trivial to write, the running of it is where all the value is, and a passing check on its own is worth almost nothing as evidence. To make the first part concrete, here is a small conformance check in R. To make the second part concrete, here is what it is missing.

A check is just a function that returns findings

A conformance rule is a question asked of a dataset that returns the rows where the answer is no. That is it. Here are three rules that cover a surprising amount of real world trouble: a required variable is missing, a key is not unique, and a date is not valid ISO 8601. Each returns a tidy table of findings, and the empty table is the pass.

library(dplyr)
library(tibble)

# Rule 1: every required variable for the domain is present
check_required_vars <- function(data, domain, required) {
  missing <- setdiff(required, names(data))
  tibble(
    domain   = domain,
    rule     = "required-variable-present",
    severity = "error",
    detail   = missing
  )
}

# Rule 2: the key variables are unique (one row per key)
check_unique_key <- function(data, domain, key) {
  dup <- data %>% count(across(all_of(key))) %>% filter(n > 1)
  tibble(
    domain   = domain,
    rule     = "unique-key",
    severity = "error",
    detail   = if (nrow(dup) == 0) character(0)
               else paste(nrow(dup), "duplicated key value(s)")
  )
}

# Rule 3: --DTC values are ISO 8601 date or datetime (allowing partials)
check_iso8601 <- function(data, domain, dtc_var) {
  iso <- "^\\d{4}(-\\d{2}(-\\d{2}(T\\d{2}:\\d{2}(:\\d{2})?)?)?)?$"
  vals <- data[[dtc_var]]
  bad  <- vals[!is.na(vals) & nzchar(vals) & !grepl(iso, vals)]
  tibble(
    domain   = domain,
    rule     = paste0("iso8601-", dtc_var),
    severity = "error",
    detail   = unique(bad)
  )
}

Three rules, each a plain function returning a findings tibble. The pattern scales to as many rules as you care to write.

Running them is a matter of binding the findings together. A real validator does this across every domain and hundreds of rules, but the shape is the same.

# A tiny AE-like frame to check
ae <- tibble(
  STUDYID  = "PILOT01",
  USUBJID  = c("01-001", "01-001", "01-002"),
  AESEQ    = c(1, 1, 1),                 # note: duplicate key on subject 01-001
  AESTDTC  = c("2026-03-01", "2026-14-01", "2026-03-05")  # note: month 14
)

findings <- bind_rows(
  check_required_vars(ae, "AE", c("STUDYID", "USUBJID", "AESEQ", "AETERM")),
  check_unique_key(ae, "AE", c("USUBJID", "AESEQ")),
  check_iso8601(ae, "AE", "AESTDTC")
) %>% filter(nzchar(detail) | lengths(detail) > 0)

findings
#> # A tibble: 3 x 4
#>   domain rule                      severity detail
#>   <chr>  <chr>                     <chr>    <chr>
#> 1 AE     required-variable-present error    AETERM
#> 2 AE     unique-key                error    1 duplicated key value(s)
#> 3 AE     iso8601-AESTDTC           error    2026-14-01

The findings table is the whole output. Writing more rules is more of the same, which is exactly why the rules are not the hard part.

You could spend a week turning this into something that reads real SDTM, loads a rule pack, and formats a report, and it would be useful work. But notice what you would have built: a program that, given a dataset, tells you which rows it does not like today. That is necessary. It is nowhere near sufficient.

The findings table is an output. The evidence is everything that lets someone reproduce it.

Why the check is not the evidence

Picture the clean report, printed, in a binder, a year later. It says zero errors. Now ask the questions an auditor asks. Which version of the dataset did it run on. Which rule set, at which version, with which thresholds. Who ran it, and on what machine. Could someone re-run it today and get the same report. A bare report answers none of these. It is a screenshot of a moment, and it can be a screenshot of a moment that no longer exists.

The evidence is not the green tick. The evidence is the recorded run: the exact dataset it saw, identified by a hash, not a filename; the exact rule pack and its version; the person who ran it, established through authenticated sign on; the environment it ran in; and a timestamp that cannot be backdated. Capture those and the report becomes reproducible and therefore defensible. Omit them and the report is an assertion, and we have written before about why an assertion of oversight is not oversight.

Anyone can produce a clean conformance report. The question that matters is whether someone else can reproduce yours, exactly, and show who stood behind it.

What we actually build around the check

So the check is the small part, and the record around it is the system. In our platform the run is the unit that is captured, not the result. Each conformance run records the dataset it read by content hash, the rule pack and version it applied, the identity of the operator through single sign on, and the environment it executed in. The whole sequence lands in a tamper evident, hash chained audit trail, so the order of events and who caused them cannot be quietly rewritten afterwards. The findings table from the check is one field in that record, not the record itself.

This is also why we keep saying you should validate the artifact, not the author. It does not matter whether a person, a script, or an assistant produced the dataset. What you validate is the dataset, and what you keep is the evidence of the validation, bound to the exact thing that was validated. The rule in R took ten lines. The discipline around running it is the part that holds up under inspection.

If you are building conformance into a pipeline and want it to produce evidence rather than just reports, we are happy to talk it through. You can reach us here.

Pinning an R environment with renv so a result still rebuilds in three years

TECHWORKSLAB — Sat, 31 Oct 2026 01:32:54 +0000

A regulator can ask you to reproduce a result years after you filed it. Not a similar result. The same one, from the same code, to the same digits. That is a high bar for R, because an R script does not carry its dependencies with it. Run it on a machine where one package has moved a minor version and an output can change in ways that are easy to miss and hard to defend.

The renv package exists for exactly this. It gives a project its own private library and writes down, in a lockfile, the exact version and hash of every package the project uses. We treat that lockfile as part of the submission record, not as developer convenience. Here is how we set it up, what it actually guarantees, and the gap you have to close yourself.

Turning a project into a pinned project

One call bootstraps it. renv::init() creates a project local library, scans the code for the packages it loads, installs them into that private library, and writes the first renv.lock.

install.packages("renv")

# From the project root, once:
renv::init()

# From then on, after adding or updating a package:
renv::snapshot()   # record the current state into renv.lock

After init, the project installs into its own library. The global library on the machine no longer matters.

The lockfile is the whole point, so it is worth looking at. It is plain JSON, it is meant to be read, and it belongs in version control next to the code.

{
  "R": {
    "Version": "4.4.1",
    "Repositories": [
      { "Name": "CRAN", "URL": "https://packagemanager.posit.co/cran/2026-09-01" }
    ]
  },
  "Packages": {
    "admiral": {
      "Package": "admiral",
      "Version": "1.1.1",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6"
    }
  }
}

Illustrative lockfile excerpt. The version and hash together pin the exact package, not just its name.

Three years later, on a clean machine, one call rebuilds the library the script expects. renv::restore() reads the lockfile and installs each package at the recorded version, verifying the hash as it goes.

# On the machine that has to reproduce the result:
renv::restore()   # installs every package at the version and hash in renv.lock

# Confirm the environment matches the lockfile before you trust an output
renv::status()

renv locks the top of the stack. The bottom of it is your job.

Two gaps renv does not close on its own

renv is excellent at what it does, and it is important to be precise about what it does not do, because the gaps are where false confidence lives.

It does not install R itself. The lockfile records that the project was built under, say, R 4.4.1, but renv::restore() will not put R 4.4.1 on the machine for you. If you restore under a different R version, a package may build differently or refuse to install. So the R version has to be pinned by whatever stands up the machine, not by renv.

By default it does not pin where packages come from in time. If your repository is the live CRAN, then a package that was yanked or superseded can make a restore fail or drift. The fix is to point the project at a dated snapshot repository, so that the restore always sees CRAN as it looked on one fixed day.

# Pin the repository to a fixed date, so a restore sees a frozen CRAN
options(repos = c(
  CRAN = "https://packagemanager.posit.co/cran/2026-09-01"
))
renv::snapshot()   # the dated URL is now recorded in renv.lock

renv pins the packages. A dated repository pins time. A container pins everything below them. You need all three before you can say a result rebuilds.

Closing the last gap with a container

The way we actually guarantee a rebuild is to combine renv with a container built on a fixed base image, for example a versioned Rocker image that carries a specific R version and operating system. The container fixes R and the system libraries; renv fixes the packages; the dated repository fixes where those packages came from. The image is built once, tagged, and kept. Reproducing the result three years on means pulling the tagged image and running renv::restore() inside it, and the answer comes back the same because nothing underneath it was free to move.

This is the practical core of what we wrote about more generally in reproducing the environment, not the answer and in standing up a validated computing environment. For deterministic R code, reproducing the environment is reproducing the answer. renv is the piece of that stack you can adopt this afternoon, and it earns its place long before anyone asks you to prove it.

If you are setting up a reproducible R environment for regulated work and want a second pair of eyes on where the gaps are, we are happy to help. You can reach us here.

Rebuilding the CDISC pilot ADSL with admiral, and reading it like a reviewer

TECHWORKSLAB — Sat, 31 Oct 2026 01:32:45 +0000

There is a good public dataset for practising this. The CDISC SDTM/ADaM pilot project shipped a small but complete set of SDTM domains, and the R package pharmaversesdtm makes them available as data frames. It is synthetic, it is open, and it is the same data the admiral documentation builds its own examples on. So we can walk through deriving a subject level analysis dataset, ADSL, from it, and nothing here depends on a client study or a number we are not allowed to show.

We want to be careful about what this is. It is a method walkthrough, written the way we approach a build, not a benchmark. We are not going to claim a match percentage against a reference ADSL, because a match percentage without the full provenance of both sides is exactly the kind of figure we argue against. What we will do is build ADSL from the pilot data, and then read the result the way a reviewer reads it.

Setup

Three packages. admiral for the derivations, dplyr for the data manipulation grammar admiral is built on, and pharmaversesdtm for the pilot SDTM data.

install.packages(c("admiral", "dplyr", "pharmaversesdtm"))

library(admiral)
library(dplyr)
library(pharmaversesdtm)

# The public CDISC pilot SDTM domains, as shipped in pharmaversesdtm
data("dm")   # demographics
data("ds")   # disposition
data("ex")   # exposure

# One row per subject in DM; this is the spine ADSL is built on
dm %>% count()  # number of subjects in the pilot DM

The pilot SDTM domains load as plain data frames, so the whole walkthrough runs on a laptop.

The spine: start from DM

ADSL has one row per subject, and DM already has one row per subject, so DM is the spine. The planned and actual treatment variables come straight across from the SDTM arm variables. This is a predecessor origin, and it is worth saying so out loud now, because origin is the first thing a reviewer will check later.

adsl <- dm %>%
  select(-DOMAIN) %>%
  mutate(
    TRT01P = ARM,        # planned treatment, predecessor of DM.ARM
    TRT01A = ACTARM      # actual treatment, predecessor of DM.ACTARM
  )

Treatment start and end, derived from exposure

Treatment start date is not collected, it is derived, and the derivation is the first place real care is needed. We take the first dosing record where the subject actually received drug, impute a missing day to the first of the month so partial dates do not drop out, and carry the datetime onto ADSL. admiral has a dedicated function for the merge so we do not hand roll a join and a de-duplication.

# Impute partial exposure start dates to the month level, then build a datetime
ex_ext <- ex %>%
  derive_vars_dtm(
    dtc = EXSTDTC,
    new_vars_prefix = "EXST",
    highest_imputation = "M"
  )

adsl <- adsl %>%
  derive_vars_merged(
    dataset_add = ex_ext,
    filter_add = (EXDOSE > 0 |
      (EXDOSE == 0 & grepl("PLACEBO", EXTRT))) & !is.na(EXSTDTM),
    new_vars = exprs(TRTSDTM = EXSTDTM),
    order = exprs(EXSTDTM, EXSEQ),
    mode = "first",
    by_vars = exprs(STUDYID, USUBJID)
  ) %>%
  derive_vars_dtm_to_dt(source_vars = exprs(TRTSDTM))  # TRTSDT, a date from the datetime

The filter is the derivation. It encodes a clinical rule: a zero dose still counts as treatment start for placebo.

The same pattern, with mode = "last" and the exposure end date, gives treatment end. Once start and end exist, treatment duration is a single admiral call rather than a date arithmetic expression a reviewer has to verify by hand.

adsl <- adsl %>%
  derive_vars_duration(
    new_var = TRTDURD,
    start_date = TRTSDT,
    end_date = TRTEDT   # derived with the same merge pattern, mode = "last"
  )

Groupings the tables will need

Age group and a few population flags are not interesting statistically, but they are where silent errors hide, because an off by one in a cut point moves subjects between rows of a demographics table without erroring. Writing the cut explicitly, and keeping it next to the data it classifies, is the honest way to do it.

adsl <- adsl %>%
  mutate(
    AGEGR1 = case_when(
      AGE < 65            ~ "<65",
      AGE >= 65 & AGE <= 80 ~ "65-80",
      AGE > 80            ~ ">80",
      TRUE                ~ NA_character_
    ),
    SAFFL = if_else(!is.na(TRTSDT), "Y", "N")  # safety population: took at least one dose
  )

Illustrative. The pilot ADSL assembled from DM and EX through named admiral derivations.

Now read it like a reviewer

Building the dataset is the easy half. The half that matters is whether the result survives someone asking hard questions of it. When we open an ADSL we did not write, these are the first checks we run, and they are the same ones we run on our own.

One row per subject, no duplicates. adsl %>% count(USUBJID) %>% filter(n > 1) should return zero rows. If it does not, a merge somewhere fanned out and the whole dataset is suspect.
Treatment start is not after treatment end. A single filter(TRTSDT > TRTEDT) finds date logic that reversed on a partial date imputation. It is a two line check that catches a class of error no table would reveal.
Population flags are internally consistent. A subject with SAFFL == "Y" and a missing TRTSDT is a contradiction, because the flag was defined from that very date.
Every derived variable can name its method. For each column, we should be able to point to the admiral call that produced it or the SDTM variable it came from. Where we cannot, the variable is not ready, however plausible its values look.

# The duplicate-key check every ADSL has to pass
dups <- adsl %>% count(USUBJID) %>% filter(n > 1)
stopifnot(nrow(dups) == 0)

# The date-order check
bad_dates <- adsl %>% filter(!is.na(TRTSDT), !is.na(TRTEDT), TRTSDT > TRTEDT)
stopifnot(nrow(bad_dates) == 0)

These are not conformance checks in the Pinnacle 21 sense. They are the sanity checks that should pass before conformance checking is even worth running.

The question is never whether the numbers look reasonable. It is whether you can say, for each one, exactly how it was produced and show the step that produced it.

Why we work this way

A monolithic ADSL program can produce the same columns. What it cannot do easily is answer the reviewer questions above, because the logic for any one variable is tangled with the logic for the rest. Building from named derivations, as we described in building ADaM datasets with the admiral package, means traceability is a property of the structure rather than something reconstructed afterwards. On the public pilot data that is a convenience. On a live submission it is the difference between a smooth review and a long one.

If you want to pressure test your own ADSL build against this kind of reading, or move a monolithic program onto a derivation library, we are happy to help. You can reach us here.

The rules we build regulated analytics tooling under

TECHWORKSLAB — Sat, 31 Oct 2026 01:31:10 +0000

Most writing about AI in regulated work argues about capability. Can a model map a dataset, write a derivation, draft a define file. We find the capability question less interesting than the constraint question, because the constraints are what decide whether any of it is usable in a filing. So here are the rules we actually build Norva under. They are not aspirations. They are the lines we do not cross, and every feature we ship has to live inside them.

Rule one: the AI never sees raw data

A model does not need a patient record to help build a derivation. It needs the structure of the data, the variable names, the types, the codelists, the specification. So that is all it gets. The assistive layer works against metadata and specifications, not against the clinical values themselves. The values are processed by validated, deterministic code that runs in an isolated environment with the data mounted read only.

This is partly a privacy position and partly an engineering one. If the model never touches a subject value, then a whole category of risk simply does not exist: there is no raw data in a prompt, no raw data in a log, no raw data in a place it was not meant to be. We wrote separately about why telling an agent not to open the folder is not a control. The short version is that isolation has to be enforced by the platform, not requested in an instruction.

The model works on the shape of the data. The deterministic code works on the data.

Rule two: every derivation ships with its validation and a risk score

A derivation that arrives without evidence is not finished, no matter who or what produced it. In our model, the unit of work is not a line of code. It is the derivation plus the test that exercises it, plus a record of what it was checked against, plus a risk score that says how much scrutiny it still needs. The three travel together. You cannot promote the code and leave the evidence behind.

The risk score is the part people ask about most, so to be plain about it: it is an indicator, not a verdict. It combines things we can measure, such as whether the derivation is a standard pattern or bespoke study logic, whether it touches a variable that feeds a primary endpoint, and whether its inputs changed since the last run. A high score does not block anything. It directs attention. It tells a reviewer which twenty derivations in a study of hundreds are the ones to read first. We covered the broader idea in FMEA risk scoring for a submission package; the rule here is simply that the score is attached at the point of creation, not bolted on at the end.

Evidence that is generated with the work is cheap. Evidence reconstructed after the fact is expensive, and it is the first thing an auditor distrusts.

Rule three: a qualified human is the last gate

Nothing becomes a controlled artifact because a model produced it. It becomes a controlled artifact because a qualified person reviewed it and attested to it, and that attestation is recorded. We call the moment this happens the freeze point. Upstream of it, the assistive layer can draft, suggest, and iterate freely. Downstream of it, the record is fixed and change is controlled.

The attestation is not a checkbox. It is a scoped electronic signature under 21 CFR Part 11, tied to the specific artifact and the specific version, with the reviewer identified through single sign on and separation of duties enforced so that the person who built a thing is not the only person who approves it. The signature, the identity, and the state change all land in a tamper evident, hash chained audit trail, so the sequence of who did what, and in what order, cannot be quietly rewritten later.

What these rules cost, and why we keep them

They cost speed, in the narrow sense. A system that let a model touch raw data, skip the attached evidence, and self approve its output would demo faster and feel more magical. It would also be unusable in a filing, because none of the questions a reviewer asks would have answers. Where did this value come from. Who checked it. How do we know the thing that ran is the thing that was validated. The rules exist so that those questions always have an answer that was captured at the time, not assembled under pressure during an inspection.

We think this is the honest shape of AI in regulated analytics. The model is assistive and human in the loop. It makes a skilled person faster at producing work that still has to meet the same standard it always did. The rules are what keep the speed from becoming a liability.

If you are drawing your own version of these lines for a validated environment, we are happy to compare notes. You can reach us here.