DEV Community: Matheus de Camargo Marques

Bibliography — PON + Smart Brewery dev.to series (EN drafts)

Matheus de Camargo Marques — Fri, 20 Mar 2026 17:28:41 +0000

Bibliography — PON + Smart Brewery dev.to series (EN drafts)

Normalized references used across docs/devto/en/. When publishing on dev.to, prefer stable URLs; verify DOIs and publisher pages periodically.

Paradigms and architecture

Author(s)	Year	Title	Where
Simão, J. M.; Borges, M. R.; Ebina, R.; Tacla, C. A.; Stadzisz, P. C.; Banaszewski, R. F.	2013	Notification Oriented Paradigm (NOP) and Imperative Paradigm: A Comparative Study	SCIRP / IJSEA — academic treatment of NOP (closely related to “notification-oriented” rule structuring; this repo uses PON as shorthand).
Cockburn, A.	2005	Hexagonal architecture (ports and adapters)	alistair.cockburn.us
Fowler, M.	2015	Ports and Adapters / Hexagonal architecture (summary)	martinfowler.com

Erlang, Elixir, OTP

Author(s)	Year	Title	Where
Armstrong, J.	2003	Making reliable distributed systems in the presence of software errors (PhD thesis)	erlang.org
Cesarini, F.; Thompson, S.	2016	Programming Erlang (2nd ed.)	Pragmatic Bookshelf — OTP design, processes, supervision.
Elixir documentation	—	`GenServer`, `Registry`, `Supervisor`, `Macro`, `@behaviour`	hexdocs.pm/elixir
Mix profiling tasks	—	`mix profile.cprof`, `eprof`, `fprof`, `tprof`	hexdocs.pm/mix/Mix.Tasks.Profile.Cprof.html and sibling task modules

Phoenix ecosystem

Author(s)	Year	Title	Where
Phoenix LiveView	—	Guides and API	hexdocs.pm/phoenix_live_view
Phoenix PubSub	—	API	hexdocs.pm/phoenix_pubsub

Data pipelines and observability

Author(s)	Year	Title	Where
Broadway	—	Documentation	hexdocs.pm/broadway
GenStage	—	Documentation	hexdocs.pm/gen_stage
Telemetry	—	`telemetry` for Erlang/Elixir	hexdocs.pm/telemetry
Timescale, Inc.	—	TimescaleDB documentation	docs.timescale.com

Analytics and BI

Author(s)	Year	Title	Where
Kimball, R.; Ross, M.	2013	The Data Warehouse Toolkit (3rd ed.) — dimensional modeling	Wiley — star schema, facts/dimensions.
Microsoft	—	Power BI REST APIs (push datasets, etc.)	Microsoft Learn — Power BI REST
PostgreSQL Global Development Group	—	`GRANT`, roles, row security	postgresql.org/docs

Digital twins and industrial context

Author(s)	Year	Title	Where
Grieves, M.	2014	Digital Twin: Manufacturing Excellence Through Virtual Factory Replication	Often cited white paper on digital-twin vocabulary (search publisher copy; ISBN/institutional PDFs vary).

Machine learning lifecycle

Author(s)	Year	Title	Where
Breck, E.; et al.	2017	The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction	research.google
Elixir Numerics	—	Axon, Scholar (Neural Network / traditional ML in Elixir)	hexdocs.pm/axon, hexdocs.pm/scholar

Queues, performance, back-pressure

Author(s)	Year	Title	Where
Little, J. D. C.	1961	A Proof for the Queuing Formula L = λW	Operations Research — foundation for Little’s Law (relation of queue length, arrival rate, wait); Wikipedia summary with citation to original.
Erlang docs	—	`:erlang.process_info/2` (mailbox size, etc.)	erlang.org/doc/man/erlang.html

This repository (deep dives in Portuguese or internal)

Topic	Path
Performance, profilers, env matrix	`docs/performance-dev.md` — EN series walkthrough: Part 11 on dev.to — Dev profiling… · repo draft
Memory pressure heuristics	`docs/memory-pressure-heuristics.md`
Message storm mitigation	PT: `docs/artigos/19_mitigacao_message_storm_pon_elixir_smart_brewery.md` — EN series: Part 10 on dev.to — When notifications explode… · repo draft
ML practical guide (PT)	`docs/artigos/27_guia_pratico_treino_ml_smart_brewery.md`
Power BI / realtime notes	`docs/power-bi-realtime.md`

Retrospective: lessons from building a reactive rules engine in Elixir

Matheus de Camargo Marques — Fri, 20 Mar 2026 17:26:49 +0000

If this helped you, you can support the author with a coffee on dev.to.

Retrospective: lessons from building a reactive rules engine in Elixir

Part 12 of 12 — This post closes the arc that started with why the BEAM fits reactive rules and ends with how we measure them under load. Part 11 on dev.to — Dev profiling: CPU, memory, and what changed after optimizations · repo draft was about profilers and reproducible workloads; here I zoom out: integrations that paid off, costs we accepted, and what I would try next if I were green-fielding tomorrow.

This is not a substitute for the earlier technical posts—treat it as a map and a checklist for your own PON-style systems. Each part now ends with a References and further reading section; the consolidated list lives in Bibliography on dev.to — PON + Smart Brewery series (EN drafts) · repo draft.

What the twelve parts built

Stretch	Idea
1–2	Notifications as the organizing principle; Facts, Rules, and Premises as OTP processes and `Registry` topics.
3–4	Less boilerplate via `defrule` / `defpremissa`; ports and adapters so rules do not hard-code IO.
5–6	Smart Brewery as a serious lab; LiveView as the operator’s window with batching and streams.
7–8	Broadway / GenStage, TimescaleDB, star-schema BI and safe read-only roles.
9	ML export / import and `ml_predictions` without blocking the engine.
10–11	Part 10 on dev.to — message storms, dedup and fan-out coalescence; Part 11 on dev.to — CPU/memory profiling with `PipelineWorkload`.

Together they describe one opinionated path: reactive core, async edges, honest measurement.

What worked well

Process boundaries match mental boundaries. A Fato is a named mailbox with a value; a Regra subscribes and reacts. That maps cleanly to drawings on a whiteboard and to Elixir supervision.

Explicit message shapes. Consumers understand {:notificacao, name, value} and {:notificacoes_lote, map}—whether they live in tec0301_pon or in the Phoenix app. Ambiguity is expensive at scale.

Two applications, one story. Keeping tec0301_pon as the engine and simulacoes_visuais as warehouse + UI avoided turning the rule engine into an Ecto or HTTP library, while still shipping a full twin demo.

Batching at the pain points. LiveView batchers, Broadway batchers, Fanout.atualizar_lote/1, and async DB writers share the same philosophy: coalesce before fan-out or disk.

Telemetry as a spine, not an afterthought. From rule firings to Broadway flushes, the codebase repeatedly uses :telemetry.execute/3 and named events so you can correlate “the twin moved” with “the UI updated” and “rows landed in TimescaleDB.” That does not replace profilers, but it anchors them: when CPU spikes, you want a span or counter that says which stage grew.

Trade-offs we accepted

DSL complexity. Macros that generate rule modules are powerful and testable, but they are a onboarding cliff. Teams that fear metaprogramming will push for YAML or database rules—each has its own cost.

Simulation vs. fidelity vs. CPU. Monte Carlo noise, strict === deduplication, and float quantization are linked: quieter logs mean less work, but also less “physical” continuity unless you design for it.

Pragmatic coupling in examples. Rules that call Adapters.PredioIO directly are easy to read; “production-shaped” indirection via Application.get_env/3 is more wiring. The series kept both stories visible.

Operational surface. TimescaleDB, CAGGs, retention, Power BI roles, and ML export paths are optional but real: the default mix phx.server story is heavier than a pure library.

English drafts, Portuguese publication layer. Keeping docs/devto/en/*.md as the paste-ready source while devto_serie_pon_smart_brewery.md tracks PT titles and dev.to slugs adds a bit of bookkeeping. The upside is a single technical narrative in one language and localized packaging when you hit publish.

What I would try earlier next time

Profiling harness from week one — PipelineWorkload-style reproducible ticks save arguments about regressions.
One canonical env matrix — document SIMULACOES_TSDB_ENABLED, Monte Carlo interval, and Broadway batch sizes in a single table (the repo now spreads this across performance-dev.md and config; still easy to drift).
Stable join keys — align persisted rule_events.regra_id with dimension tables (r_01 vs "1") in one migration or view, so BI and ML exports do not need ad hoc bridges.
Stricter “no work if unchanged” policy — extend the dedup story to any hot path that allocates (e.g. telemetry encode) once profiling proves it matters.

These are hypotheses, not promises—verify on your workload.

Code spine (the same system, twelve lenses)

Part 2 / 10 — fact update and dispatch only when the value changes:

# lib/tec0301_pon/pon/fato.ex (concept)
def handle_cast({:atualizar, novo_valor}, estado) do
  if valor_igual?(estado.valor, novo_valor) do
    {:noreply, estado}
  else
    # … update state, ETS, Registry.dispatch → {:notificacao, nome, valor}
    {:noreply, novo_estado}
  end
end

Part 3 — DSL-shaped rule:

defrule RegraIrrigacao,
  watch: [:temp_ambiente, :umidade_solo, :estado_bomba, :nivel_tanque],
  when: memoria[:temp_ambiente] > 30 and memoria[:umidade_solo] < 40,
  do: (
    Adapters.BombaDeAgua.ligar()
    Tec0301Pon.PON.Fato.atualizar(:estado_bomba, :ligada)
  )

Part 4 — port = behaviour:

defmodule Tec0301Pon.Ports.PredioAtuadores do
  @callback ligar_luz() :: :ok
  @callback trancar_porta() :: :ok
end

Part 6 — operator route:

live "/smart-brewery", SmartBreweryLive, :index
live "/smart-brewery/ml-predictions", MlPredictionsLive, :index

Part 7 — Broadway flush to PubSub and optional TSDB:

if Application.get_env(:simulacoes_visuais, :tsdb_enabled, false) do
  SimulacoesVisuais.SmartBrewery.TelemetryAsyncWriter.cast_batch(list)
end

Part 9 — export and import (shell):

mix export.ml --out /tmp/ml_export --since-hours 168
mix import.ml.predictions --file /tmp/preds.jsonl

[Part 11 on dev.to](https://dev.to/matheuscamarques/dev-profiling-cpu-memory-and-what-changed-after-optimizations-28hb) — reproducible load:

mix profile.cprof -e "SimulacoesVisuais.Profile.PipelineWorkload.run()"

End-to-end picture

flowchart LR
  subgraph engine [tec0301_pon]
    F[Fato]
    R[Regra]
    A[Adapters]
  end
  subgraph app [simulacoes_visuais]
    MC[MonteCarlo]
    BW[Broadway]
    LV[LiveView]
    TSDB[(TimescaleDB)]
  end
  MC --> F
  F --> R
  R --> A
  F --> BW
  BW --> TSDB
  BW --> LV

Series index (parts 1–12 + consolidated bibliography)

Part	Title	Draft / published
1	Notification-Oriented Paradigm (PON) in Elixir: why the BEAM fits reactive rules	dev.to · repo draft
2	From whiteboard to code: mapping Facts, Rules, and Premises to OTP processes	dev.to · repo draft
3	A metaprogrammed DSL: `defrule` and `defpremissa` with less PON boilerplate	dev.to · repo draft
4	Hexagonal architecture + PON: Ports & Adapters to decouple the engine	dev.to · repo draft
5	Smart Brewery: a digital twin brewery as a PON lab	dev.to · repo draft
6	Phoenix LiveView in real time: an operations UI on top of a rules engine	dev.to · repo draft
7	From simulation to storage: telemetry, Broadway/GenStage, and TimescaleDB	dev.to · repo draft
8	BI without mystery: dimensions, facts, and consuming the data (e.g. Power BI)	dev.to · repo draft
9	ML on the digital twin: export, train pilots, and import predictions back into the app	dev.to · repo draft
10	When notifications explode: message storms, deduplication, and back-pressure in PON	dev.to · repo draft
11	Dev profiling: CPU, memory, and what changed after optimizations	dev.to · repo draft
12	Retrospective: lessons from building a reactive rules engine in Elixir	this post
13	Bibliography — PON + Smart Brewery dev.to series (EN drafts)	dev.to · repo draft

Portuguese titles and publication notes for dev.to live in docs/devto_serie_pon_smart_brewery.md.

References and further reading (series-level)

Master bibliography — normalized table of books, papers, and docs used across Parts 1–12: Bibliography on dev.to — PON + Smart Brewery series (EN drafts) · repo draft.
NOP / PON — Simão et al. (2013) comparative study — SCIRP.
OTP / BEAM — Armstrong (2003) thesis — PDF; Cesarini & Thompson, Programming Erlang.
Hexagonal — Cockburn — ports and adapters.
Data + time-series — Kimball & Ross (dimensional modeling); TimescaleDB — docs.timescale.com.
ML readiness — Breck et al., ML Test Score — Google Research.

Thank you

If you followed from Part 1 on dev.to: you have seen one way to marry reactive rules, OTP, and industrial-style twins—not the only way. Take the patterns (boundaries, batching, measurement) and adapt the machinery to your domain.

End of series.

Previous: Part 11 on dev.to — Dev profiling: CPU, memory, and what changed after optimizations · repo draft

Code map (monorepo root): lib/tec0301_pon/, apps/simulacoes_visuais/, docs/performance-dev.md, docs/devto_serie_pon_smart_brewery.md.

Dev profiling: CPU, memory, and what changed after optimizations

Matheus de Camargo Marques — Fri, 20 Mar 2026 17:22:45 +0000

If this helped you, you can support the author with a coffee on dev.to.

Dev profiling: CPU, memory, and what changed after optimizations

Part 11 of 12 — Part 10 on dev.to — When notifications explode: message storms, deduplication, and back-pressure in PON · repo draft described deduplication, fan-out batching, and mailbox draining in the PON core. Those changes are meant to reduce wasted work—but you only know they helped if you measure the same workload twice under the same knobs.

This post is a practical guide to dev profiling in this monorepo: reproducible load via SimulacoesVisuais.Profile.PipelineWorkload, Mix tasks built on OTP profilers, and how to read results without fooling yourself. Deep dives and longer checklists live in docs/performance-dev.md and docs/memory-pressure-heuristics.md. Part 12 closes the series with a retrospective.

What profilers actually show

On the BEAM, mix profile.cprof, eprof, fprof, and (OTP 27+) tprof answer different questions (official task docs under Mix.Tasks.Profile on HexDocs; flags and caveats in this repo are expanded in performance-dev.md):

Tool	Rough question
cprof	Which functions ran most often?
eprof	Where did this process spend time?
fprof	What does the call graph look like in time?
tprof	Aggregated time, calls, or allocation (WORDS) across processes

None of them is a direct “heap map.” RAM pressure still needs :erlang.memory/0, Process.info/2, Observer, or LiveDashboard VM metrics—hot code paths that allocate a lot often correlate with high call counts, but correlation is not identity.

Reproducible load: `PipelineWorkload`

The module SimulacoesVisuais.Profile.PipelineWorkload runs Monte Carlo ticks through the same stack as production-style simulation: PON facts, hybrid models, telemetry fan-out, and (if enabled) TSDB writers.

# SimulacoesVisuais.Profile.PipelineWorkload.run/1 (opening)
def run(opts \\ []) when is_list(opts) do
  duration_ms = Keyword.get(opts, :duration_ms) || env_duration_ms()
  ticks = Keyword.get(opts, :ticks, env_int("PROFILE_PIPELINE_TICKS", 30))
  max_ticks = Keyword.get(opts, :max_ticks, env_int("PROFILE_PIPELINE_MAX_TICKS", 5_000_000))
  mode = Keyword.get(opts, :mode, mode_from_env())
  memory? = Keyword.get(opts, :memory, env_bool("PROFILE_PIPELINE_MEMORY", false))

  {:ok, _} = Application.ensure_all_started(:simulacoes_visuais)
  MonteCarlo.stop_loop()
  if memory?, do: print_memory_section("before")
  # … duration_ms branch → run_until_deadline_* OR fixed tick count …
  if memory?, do: print_memory_section("after")
  :ok
end

Two modes matter for interpretation

:via_genserver (default) — calls SmartBreweryMonteCarlo.run_tick_sync/0. Realistic for end-to-end behavior; some profilers attribute time to the caller (e.g. Mix) as much as to the GenServer body.
:in_process — PROFILE_PIPELINE_MODE=in_process runs run_tick_pure/1 in the current process. Better for fprof-style call graphs; does not advance the live GenServer RNG state—profiling only.

Set duration instead of a fixed tick count with PROFILE_PIPELINE_DURATION_MS (milliseconds wall clock), capped by PROFILE_PIPELINE_MAX_TICKS.

Environment variables (cheat sheet)

Variable	Role
`PROFILE_PIPELINE_TICKS`	Fixed number of ticks when duration is off
`PROFILE_PIPELINE_DURATION_MS`	Run until wall-clock deadline
`PROFILE_PIPELINE_MAX_TICKS`	Safety ceiling for duration mode
`PROFILE_PIPELINE_MODE`	`via_genserver` (default) or `in_process`
`PROFILE_PIPELINE_MEMORY`	`1` / `true` → print `:erlang.memory/0` + key processes before/after
`SIMULACOES_TSDB_ENABLED`	Match your DB-on vs DB-off scenario
`LOGGER_LEVEL`	Often `warning` to reduce log noise during long runs

For before/after comparisons, also lock Monte Carlo and pipeline tuning: MONTE_CARLO_INTERVAL_MS, MONTE_CARLO_FACTS_PER_TICK_MIN / MAX, TELEMETRY_PIPELINE_BATCH_SIZE, TELEMETRY_PIPELINE_BATCH_TIMEOUT_MS, TELEMETRY_PRODUCER_MAX_QUEUE, OEE_PUBSUB_MIN_INTERVAL_MS—see the tables in performance-dev.md.

Quick commands

Call-count hotspot (cprof), TSDB off, long window:

cd apps/simulacoes_visuais
mkdir -p tmp/profile
PROFILE_PIPELINE_DURATION_MS=120000 LOGGER_LEVEL=warning \
  SIMULACOES_TSDB_ENABLED=false \
  mix profile.cprof -e "SimulacoesVisuais.Profile.PipelineWorkload.run()" \
  | tee tmp/profile/cprof-sample.txt

Narrow to one module:

mix profile.cprof --module SimulacoesVisuais.SmartBreweryMonteCarlo \
  -e "SimulacoesVisuais.Profile.PipelineWorkload.run()"

Memory snapshot without a profiler:

PROFILE_PIPELINE_MEMORY=1 PROFILE_PIPELINE_TICKS=30 \
  mix profile.pipeline
# alias for: mix simulacoes_visuais.profile_workload

OTP 27+ aggregated time:

PROFILE_PIPELINE_DURATION_MS=120000 LOGGER_LEVEL=warning \
  mix profile.tprof --type time --report total \
  -e "SimulacoesVisuais.Profile.PipelineWorkload.run()"

fprof trace file — put -e "..." before --trace-to-file, or Mix mis-parses the path:

mix profile.fprof -e "SimulacoesVisuais.Profile.PipelineWorkload.run()" \
  --trace-to-file tmp/profile/run.trace \
  | tee tmp/profile/fprof-out.txt

60-second battery

From the repo root, scripts/run_profile_60s.sh runs several profiler variants and writes under apps/simulacoes_visuais/tmp/profile/*-60s.txt. Useful when you want comparable artifacts after a change without hand-typing flags.

Profiling the Monte Carlo GenServer itself

With PipelineWorkload in default via_genserver mode, eprof attached to the Mix process often shows GenServer.call / client time—not the full body of SmartBreweryMonteCarlo. To attribute work to the right OTP process:

Start the app (mix phx.server or Application.ensure_all_started(:simulacoes_visuais)).
Resolve PIDs: Process.whereis(SimulacoesVisuais.SmartBreweryMonteCarlo), and when TSDB is on, writers such as RuleEventWriter.
Attach eprof / fprof to those PIDs from an iex session on the same node (see OTP profiling docs), while generating load from another shell or the UI.

Cross-check reductions and mailbox length in Observer or LiveDashboard while PROFILE_PIPELINE_MEMORY=1 prints heap and queue depth for the registered names the workload already tracks.

Baselines: “light” vs “heavy” server

Before blaming PON, confirm whether you are in a quiet or noisy dev profile. performance-dev.md suggests:

# Lighter: no TSDB, no auto Monte Carlo
PHX_LV_DEBUG=0 SIMULACOES_TSDB_ENABLED=false AUTO_START_MONTE_CARLO=false mix phx.server

# Heavier: TSDB, fast MC, more LiveView debug
PHX_LV_DEBUG=1 SIMULACOES_TSDB_ENABLED=true AUTO_START_MONTE_CARLO=true \
  MONTE_CARLO_INTERVAL_MS=500 mix phx.server

Profiling workloads should use the same economic assumptions as the hypothesis you are testing.

Optimizations this repo ties to measurements

Internal write-ups (Portuguese) connect architecture to numbers: article 18 (runtime), article 19 (message storm mitigation), article 20 (ETS write_concurrency, Registry partitions, Rete spike, Broadway tuning). The checklist at the top of performance-dev.md maps each item to modules—for example: Registry partitions in Tec0301Pon.Application, ETS read_concurrency / write_concurrency on fact values, Broadway :telemetry_pipeline_processor_concurrency / batcher settings in config, Fanout switching to Task.async_stream when the update map has more than four pairs, and optional Rete experiments via Tec0301Pon.PON.ReteSpike. Profiling tells you which row in that table actually matters for your tick rate and hardware.

Do not publish a single “we saved 40%” headline unless you have paired runs (same commit range, same env, same PROFILE_PIPELINE_*) and you document where the files live—otherwise you are storytelling, not benchmarking.

BI / TSDB sanity (one paragraph)

If charts look empty while telemetry_events has millions of rows, check time windows: mix verify.bi uses small LIMIT samples and often last 24h in SQL; mix verify.tsdb reports global counts and MAX(ts). Misaligned windows are a data issue, not a broken query—see performance-dev.md § BI native.

Flow under profiling

flowchart LR
  PW[PipelineWorkload]
  MC[SmartBreweryMonteCarlo]
  PON[PON Fato Regra]
  BW[Broadway Telemetry]
  W[AsyncWriters optional]
  Mix[Mix process profiler]
  PW --> MC
  MC --> PON
  PON --> BW
  BW --> W
  Mix -->|"via_genserver: often attributes client time"| MC
  Mix -->|"in_process: run_tick_pure only"| MC

Summary

Part 10 reduced redundant messages; Part 11 gives you the bench: PipelineWorkload, Mix profilers, memory snapshots, and explicit env discipline. Treat every optimization as a hypothesis until two traces agree. Part 12 zooms out with a series retrospective and a full index of posts.

References and further reading

Mix profiling — profile.cprof, profile.eprof, profile.fprof, profile.tprof — HexDocs (cprof) and sibling tasks.
Armstrong (2003) — process isolation and failure domains — thesis PDF (context for why mailbox/profiler stories matter).
In this repo — pipeline_workload.ex, performance-dev.md, memory-pressure-heuristics.md, run_profile_60s.sh; internal articles 18, 20. Expanded list: Bibliography on dev.to — PON + Smart Brewery series (EN drafts) · repo draft.

Published on dev.to: Dev profiling: CPU, memory, and what changed after optimizations — tracked in docs/devto_serie_pon_smart_brewery.md.

Previous: Part 10 on dev.to — When notifications explode: message storms, deduplication, and back-pressure in PON · repo draft

Next: Part 12 — Retrospective: lessons from building a reactive rules engine in Elixir — end of series (index + lessons learned).

When notifications explode: message storms, deduplication, and back-pressure in PON

Matheus de Camargo Marques — Fri, 20 Mar 2026 17:18:07 +0000

If this helped you, you can support the author with a coffee on dev.to.

When notifications explode: message storms, deduplication, and back-pressure in PON

Part 10 of 12 — Part 9 on dev.to — ML on the digital twin: export, train pilots, and import predictions back into the app · repo draft showed how to export history and import model scores. Before adding more consumers, it pays to ask: what happens when the twin’s tick rate rises and every Fato.atualizar/2 fans out to many Regra processes?

This post is about message storms on the BEAM: redundant Registry.dispatch work, rule mailboxes full of duplicate context, and back-pressure at the edges. The patterns below are implemented in tec0301_pon and exercised by Smart Brewery + simulacoes_visuais. Deeper Portuguese notes live in docs/artigos/19_mitigacao_message_storm_pon_elixir_smart_brewery.md; reproducing profiles is covered in docs/performance-dev.md. Part 11 on dev.to zooms in on profiling methodology and before/after numbers. Intuitively, stable queueing systems obey Little’s Law (mean work in system ∝ arrival rate × mean delay—Little 1961); storms push the “work in system” term through the roof unless you cut redundant arrivals or widen the bottleneck.

Symptom: CPU spent orchestrating, not deciding

In actor-style PON graphs, one fact update can mean:

A GenServer cast on the fact process.
Registry.dispatch to every subscriber.
One message per rule that watches the fact.
Condition evaluation and possibly actions—again per message.

Under Monte Carlo or dense PLC simulation, steps 2–3 multiply into a storm: schedulers stay busy copying terms between processes, even when the business outcome would be the same after coalescing updates.

1. Deduplicate at the source (`Fato`)

If the new value is strictly equal to the current one (===), skip dispatch entirely and do not bump the fact’s notification statistics:

# lib/tec0301_pon/pon/fato.ex — handle_cast/2 (excerpt)
def handle_cast({:atualizar, novo_valor}, estado) do
  if valor_igual?(estado.valor, novo_valor) do
    {:noreply, estado}
  else
    novo_estado =
      estado
      |> Map.update(:estatisticas, 1, &(&1 + 1))
      |> Map.put(:valor, novo_valor)

    ets_put(estado.nome, novo_valor)

    Registry.dispatch(Tec0301Pon.PON.PubSub, estado.nome, fn inscritos ->
      for {pid, _} <- inscritos, do: send(pid, {:notificacao, estado.nome, novo_valor})
    end)

    {:noreply, novo_estado}
  end
end

Float note: equality is exact. Noisy floats should be quantized upstream (as in the Smart Brewery random walk) if you rely on this filter—adding a global epsilon for all types would surprise rules that compare structured terms.

2. Coalesce across facts (`Fanout` + `atualizar_lote/1`)

Correlated equipment often updates several facts in the same tick (e.g. filter pressure, clarity, pump speed). Calling atualizar/2 three times means three dispatches and three bursts to shared rules. Fato.atualizar_lote/1 applies {:atualizar_sem_dispatch, val} per changed fact, then sends one {:notificacoes_lote, map} per subscriber PID:

# lib/tec0301_pon/pon/fanout.ex — notify_lote/1 (excerpt)
defp notify_lote(changed) when is_map(changed) do
  pids =
    changed
    |> Map.keys()
    |> Enum.flat_map(fn key ->
      Registry.lookup(@pubsub, key) |> Enum.map(fn {pid, _} -> pid end)
    end)
    |> Enum.uniq()

  msg = {:notificacoes_lote, changed}
  Enum.each(pids, fn pid -> send(pid, msg) end)
end

Rules merge only the keys they watch:

# lib/tec0301_pon/pon/regra.ex — after {:notificacoes_lote, updates}
relevante = Map.take(updates, estado.fatos)
nova_memoria = Map.merge(estado.memoria, relevante)
{nova_memoria, drained} = drain_notificacoes(nova_memoria, estado, 0)
# then update :estatisticas_notificacoes and call avaliar_apos_memoria/2

Large batches use Task.async_stream with bounded concurrency inside Fanout.atualizar_lote/1; very small maps update sequentially—see the module for thresholds.

3. Drain the rule mailbox before evaluating (`drain_notificacoes/3`)

Even with fan-out reduced, bursts still happen. After applying the first notification, Regra empties the mailbox of more {:notificacao, …} / {:notificacoes_lote, …} with a zero-timeout receive, then runs one avaliar_condicao/2:

# lib/tec0301_pon/pon/regra.ex — drain (excerpt)
defp drain_notificacoes(memoria, estado, acc) do
  receive do
    {:notificacao, nome, valor} ->
      drain_notificacoes(Map.put(memoria, nome, valor), estado, acc + 1)

    {:notificacoes_lote, upd} when is_map(upd) ->
      rel = Map.take(upd, estado.fatos)
      drain_notificacoes(Map.merge(memoria, rel), estado, acc + 1)
  after
    0 ->
      {memoria, acc}
  end
end

This is debounce-by-draining: the rule sees the latest memory state for the burst window, not every intermediate message.

4. Fast reads without blocking the fact (`ETS`)

Hot paths that only need the current value (e.g. filling rule memory, Monte Carlo readers) call Fato.obter/1, which prefers a public ETS table with read_concurrency and write_concurrency:

# lib/tec0301_pon/pon/fato.ex — obter/1 (excerpt)
def obter(nome_do_fato) when is_atom(nome_do_fato) do
  case :ets.lookup(@ets_table, nome_do_fato) do
    [{^nome_do_fato, valor}] -> valor
    [] -> GenServer.call(nome_do_fato, :obter)
  end
end

Tec0301Pon.Application calls Fato.ensure_ets!() before registering the Registry so lookups are safe during startup races.

5. Back-pressure outside the engine (Phoenix / Broadway)

The PON core is not the only fan-out. SmartBreweryFactBroadcaster, LiveViewEventBatcher, TelemetryProducer (bounded queue with drop-oldest), and Broadway batchers implement pressure so PostgreSQL and LiveView do not amplify storms—see Part 6 on dev.to · repo draft and Part 7 on dev.to · repo draft. SmartBreweryMonteCarlo caches Application.get_env bounds in init so the tick loop does not hit the application environment on every iteration.

Message contract cheat sheet

Message	Typical producer	Consumer
`{:notificacao, name, value}`	`Fato` after a real change	Rules, telemetry bridge
`{:notificacoes_lote, %{atom => value}}`	`Fanout.notify_lote/1`	Same subscribers; rules `Map.take/2` relevant keys

Code that only ever calls atualizar/2 keeps the single-notification shape; simulation paths that batch use atualizar_lote/1.

Measuring (preview — full walkthrough in Part 11 on dev.to)

The repo ships SimulacoesVisuais.Profile.PipelineWorkload and Mix tasks such as mix profile.cprof for repeatable runs. Example harness (TSDB off, headless):

cd apps/simulacoes_visuais
mkdir -p tmp/profile
PROFILE_PIPELINE_DURATION_MS=60000 LOGGER_LEVEL=warning \
  SIMULACOES_TSDB_ENABLED=false SIMULACOES_HEADLESS=true \
  mix profile.cprof -e "SimulacoesVisuais.Profile.PipelineWorkload.run()" \
  | tee tmp/profile/cprof-sample.txt

Treat percent gains as hypotheses until you A/B the same workload on two commits; the internal article avoids advertising fixed “30–50%” without paired measurements.

Flow diagram

flowchart TB
  subgraph reduce [Reduce fan-out]
    F[Fato atualizar]
    D{Value changed?}
    FD[Registry dispatch]
    L[Fato atualizar_lote]
    FN[Fanout notify_lote]
  end
  subgraph rule [Rule process]
    M[Mailbox]
    DR[drain_notificacoes]
    EV[avaliar_condicao once]
  end
  F --> D
  D -->|no| X[no notify]
  D -->|yes| FD
  FD --> M
  L --> FN
  FN --> M
  M --> DR --> EV

Summary

Message storms are a property of graph fan-out, not a judgment on PON. This codebase mitigates them with deduplication, batch notifications, mailbox draining, ETS reads, and bounded telemetry pipelines—without changing what a rule means, only how often it is prodded. Part 11 on dev.to ties these knobs to profiles and memory heuristics.

References and further reading

Little, J. D. C. (1961) — A proof for the queuing formula L = λW — classic Little’s Law; Wikipedia entry with citation to Operations Research.
Erlang — :erlang.process_info/2 (mailbox length, etc.) — manual.
Elixir Registry — dispatch semantics — HexDocs.
In this repo — fato.ex, fanout.ex, regra.ex, application.ex; artigo 19, performance-dev.md. Expanded list: Bibliography on dev.to — PON + Smart Brewery series (EN drafts) · repo draft.

Published on dev.to: When notifications explode: message storms, deduplication, and back-pressure in PON — tracked in docs/devto_serie_pon_smart_brewery.md.

Previous: Part 9 on dev.to — ML on the digital twin: export, train pilots, and import predictions back into the app · repo draft

Next: Part 11 on dev.to — Dev profiling: CPU, memory, and what changed after optimizations · repo draft

ML on the digital twin: export, train pilots, and import predictions back into the app

Matheus de Camargo Marques — Fri, 20 Mar 2026 17:13:39 +0000

If this helped you, you can support the author with a coffee on dev.to.

ML on the digital twin: export, train pilots, and import predictions back into the app

Part 9 of 12 — Part 8 on dev.to — BI without mystery: dimensions, facts, and consuming the data (e.g. Power BI) · repo draft gave you a star-shaped analytical surface in PostgreSQL/TimescaleDB. Machine learning needs the same history in files or arrays, then a path to bring scores back where operators already look: the Phoenix app.

This post covers mix export.ml, the CSV contract documented in MLDatasetExport, in-Beam pilot trainers (mix simulacoes_visuais.ml_train), batch import of predictions as JSONL, and the /smart-brewery/ml-predictions LiveView. Part 10 on dev.to shifts to message storms and back-pressure in the PON engine.

For a longer Portuguese walkthrough (notebooks, Python sketches), see docs/artigos/27_guia_pratico_treino_ml_smart_brewery.md.

Closed loop in three hops

Export — Snapshot telemetry, OEE, anomalies, rule events, dimensions, and optional CAGG slices to a directory of CSVs.
Train / infer offline — Use Elixir pilots for demos, or Python/R/Julia on the same files; produce JSON Lines (one JSON object per row).
Import + display — mix import.ml.predictions (alias) bulk-inserts into ml_predictions; MlPredictionsLive lists recent rows.

Training itself is intentionally out of band: the app does not need GPU drivers to be valuable as the system of record for predictions. For production readiness checklists (monitoring, data validation, CI for models), Google’s ML Test Score rubric is a compact reference (Breck et al., 2017); this repo implements only a thin slice—export/import contracts and a LiveView reader—not full MLOps.

Step 1: `mix export.ml`

From apps/simulacoes_visuais, with :tsdb_enabled and migrations applied:

mix export.ml --out /tmp/ml_export --since-hours 168
# Long form: mix simulacoes_visuais.export_ml --out /tmp/ml_export --since-hours 72
# Skip some CAGGs if missing: --no-cagg or --no-cagg-1h-1day

The task delegates to SimulacoesVisuais.MLDatasetExport.export_all/2, which writes UTF-8 CSVs with headers. The module’s moduledoc is the contract for downstream notebooks—example excerpts:

# SimulacoesVisuais.MLDatasetExport @moduledoc (SQL shapes, excerpt)
# Telemetry (multivariate series)
#   SELECT ts, fact_name, value_float, value_int, value_str, ...
#   FROM telemetry_events WHERE ts >= $1 ORDER BY ts ASC;
# OEE (regression target)
#   SELECT ts, oee_pct, availability_pct, performance_pct, quality_pct, ...
#   FROM oee_snapshots WHERE ts >= $1 ORDER BY ts ASC;
# Rule firings (discrete events)
#   SELECT ts, regra_id, case_id, ... FROM rule_events WHERE ts >= $1 ...
# Dimensions + telemetry_events_1min / _1h / _1day — see source for full SQL.

Typical filenames include telemetry_events.csv, oee_snapshots.csv, rule_events.csv, dim_equipamento_fbe.csv, and CAGG exports—matching the README’s headless simulation story (Docker Postgres, Monte Carlo, then export).

The same migration that introduced ml_predictions also adds case_id to rule_events, so exported rule traces can align with process-mining style case identifiers when you join predictions back to operational event logs.

Step 2a: Elixir pilots (Scholar / Axon)

For reproducible demos without leaving the BEAM, mix simulacoes_visuais.ml_train reads the same CSV directory:

mix simulacoes_visuais.ml_train --dir /tmp/ml_export --pilot oee
mix simulacoes_visuais.ml_train --dir /tmp/ml_export --pilot fermentation --epochs 40
mix simulacoes_visuais.ml_train --dir /tmp/ml_export --pilot anomaly --epochs 40

oee — Scholar linear regression baseline on exported OEE series.
fermentation — Axon MLP pilot on fermentation-related signals.
anomaly — Axon autoencoder focused on FBE_01-style vibration features.

The task prints metrics to the shell; production workflows usually emit JSONL from a separate inference job and import below.

Step 2b: JSONL format for import

Each line is a JSON object. model_name is required; ts is optional (defaults to “now” in UTC, normalized to microsecond precision for Ecto).

{"model_name":"oee_linear_v1","ts":"2025-03-20T12:00:00.000000Z","target_name":"oee_pct","value_float":87.4,"metadata":{"rmse":2.1,"export_window_h":168}}

Optional fields: target_name, value_float, metadata (object, stored as JSON in Postgres).

Step 3: persist predictions — schema and context

The migration creates a narrow fact table for batch scores:

# priv/repo/migrations/20260319120000_add_rule_events_case_id_and_ml_predictions.exs (excerpt)
create table(:ml_predictions, primary_key: false) do
  add :id, :binary_id, primary_key: true
  add :ts, :utc_datetime_usec, null: false
  add :model_name, :string, null: false
  add :target_name, :string
  add :value_float, :float
  add :metadata, :map
  timestamps(type: :utc_datetime_usec)
end

create index(:ml_predictions, [:ts])
create index(:ml_predictions, [:model_name, :ts])

# SimulacoesVisuais.MlPrediction (schema excerpt)
@primary_key {:id, :binary_id, autogenerate: true}
schema "ml_predictions" do
  field(:ts, :utc_datetime_usec)
  field(:model_name, :string)
  field(:target_name, :string)
  field(:value_float, :float)
  field(:metadata, :map)
  timestamps(type: :utc_datetime_usec)
end

Bulk insert from decoded JSON maps:

# SimulacoesVisuais.MlPredictions.insert_from_decoded_maps/1 (excerpt)
# Each map must include "model_name"; missing model raises ArgumentError.
rows =
  Enum.map(maps, fn m ->
    m = Map.new(m, fn {k, v} -> {to_string(k), v} end)
    %{
      id: Ecto.UUID.generate(),
      ts: parse_ts(m["ts"]) || now,
      model_name: m["model_name"],
      target_name: m["target_name"],
      value_float: m["value_float"],
      metadata: m["metadata"] || %{},
      inserted_at: now,
      updated_at: now
    }
  end)

{n, _} = Repo.insert_all(MlPrediction, rows)
{:ok, n}

Mix task: import from disk

# mix simulacoes_visuais.ml_import_predictions @moduledoc (usage)
#   mix simulacoes_visuais.ml_import_predictions --file /path/to/preds.jsonl
# Alias in mix.exs: mix import.ml.predictions --file /path/to/preds.jsonl

The task streams the file, Jason.decode!/1 per line, and calls MlPredictions.insert_from_decoded_maps/1. It requires :tsdb_enabled so Repo is part of the running app configuration you expect in TSDB workflows.

LiveView: `/smart-brewery/ml-predictions`

# SimulacoesVisuaisWeb.MlPredictionsLive — mount/3 (excerpt)
def mount(_params, _session, socket) do
  {preds, tsdb?} = load_predictions()
  {:ok, assign(socket, predictions: preds, tsdb_enabled: tsdb?)}
end

defp load_predictions do
  tsdb? = Application.get_env(:simulacoes_visuais, :tsdb_enabled, false)

  preds =
    if tsdb? do
      SimulacoesVisuais.MlPredictions.list_recent(100)
    else
      []
    end

  {preds, tsdb?}
end

Router (already introduced in Part 6 on dev.to):

live "/smart-brewery/ml-predictions", MlPredictionsLive, :index

The template surfaces timestamp, model, target, and value, plus a Refresh button—enough to validate that your batch job actually landed in the warehouse.

Flow diagram

flowchart LR
  subgraph db [TimescaleDB]
    TE[telemetry_events]
    OEE[oee_snapshots]
    RE[rule_events]
    DIM[dimensions]
    MP[ml_predictions]
  end
  subgraph offline [Offline ML]
    CSV[CSV export dir]
    PY[Python / R / etc.]
    JSONL[preds.jsonl]
  end
  subgraph beam [Elixir optional]
    PILOT[ml_train pilots]
  end
  TE --> Export
  OEE --> Export
  RE --> Export
  DIM --> Export
  Export[mix export.ml] --> CSV
  CSV --> PY
  CSV --> PILOT
  PY --> JSONL
  Import[mix import.ml.predictions] --> MP
  JSONL --> Import
  MP --> LV[MlPredictionsLive]

Summary

Part 7 on dev.to and Part 8 on dev.to built and modeled historical plant data. This post completes the ML ergonomics: standard CSV exports, documented SQL, optional BEAM-native pilots, a simple import contract, and a LiveView to read ml_predictions. Next: keeping the notification graph healthy when rates spike (Part 10 on dev.to).

References and further reading

Breck et al. (2017) — The ML Test Score — Google Research (production-readiness rubric; not a tutorial).
Axon / Scholar — neural and classical ML in Elixir — hexdocs.pm/axon, hexdocs.pm/scholar.
In this repo (PT) — docs/artigos/27_guia_pratico_treino_ml_smart_brewery.md.
In this repo (code) — export_ml task, ml_dataset_export.ex, ml_train task, ml_import_predictions task, ml_predictions.ex, ml_predictions_live.ex. Expanded list: Bibliography on dev.to — PON + Smart Brewery series (EN drafts) · repo draft.

Published on dev.to: ML on the digital twin: export, train pilots, and import predictions back into the app — tracked in docs/devto_serie_pon_smart_brewery.md.

Previous: Part 8 on dev.to — BI without mystery: dimensions, facts, and consuming the data (e.g. Power BI) · repo draft

Next: Part 10 on dev.to — When notifications explode: message storms, deduplication, and back-pressure in PON · repo draft

BI without mystery: dimensions, facts, and consuming the data (e.g. Power BI)

Matheus de Camargo Marques — Fri, 20 Mar 2026 17:09:28 +0000

If this helped you, you can support the author with a coffee on dev.to.

BI without mystery: dimensions, facts, and consuming the data (e.g. Power BI)

Part 8 of 12 — Part 7 on dev.to — From simulation to storage: telemetry, Broadway/GenStage, and TimescaleDB · repo draft landed raw rows in telemetry_events and sibling tables for OEE, anomalies, and rule firings. Analysts rarely want fifty-seven wide columns per millisecond; they want a model they can relate in a semantic layer and refresh on a schedule—or query live with guardrails.

This post describes the star-style objects in the Smart Brewery Postgres/Timescale database, how SimulacoesVisuais.SmartBreweryBI mirrors those queries for the in-app BI tab, and how Power BI (or any SQL BI tool) can connect safely. Part 9 on dev.to picks up ML exports and prediction round-trips. The dimensional vocabulary—facts vs dimensions, conformed keys, slowly changing attributes—follows the Kimball-style methodology (The Data Warehouse Toolkit, 3rd ed., Wiley); our schema is a pragmatic subset for a twin demo, not a full enterprise bus matrix.

Why not query only `telemetry_events`?

The hypertable is the source of truth for high-volume numeric telemetry. For dashboards you usually:

Bucket time with time_bucket (TimescaleDB).
Aggregate (avg, min, max) per bucket and signal.
Join to dimensions so charts show “Fermentador A” instead of fbe_06_internal_temp.

Continuous aggregates (telemetry_events_1min, _1h, _1day) precompute that roll-up. Views on top expose a fact shape with fbe_id and fact_name ready for foreign-key-style relationships in Power BI.

Dimension tables: equipment and variables

The migration seeds two conformed dimensions used across reports:

# priv/repo/migrations/20250319200000_add_star_schema_dimensions_and_fact_views.exs (excerpt)
create table(:dim_equipamento_fbe, primary_key: false) do
  add :fbe_id, :string, primary_key: true
  add :nome, :string, null: false
  add :fase_operacional, :string
end

# INSERT … FBE_01 … FBE_11 with Portuguese operational labels

create table(:dim_variaveis_mapeamento, primary_key: false) do
  add :fact_name, :string, primary_key: true
  add :descricao, :string, null: false
  add :unidade, :string
end

# INSERT … maps each fbe_XX_* fact atom string to human description + unit

A later migration adds descricao_longa (aligned with FatoDescriptions) and dim_regras: metadata for PON rules (r_01 … r_12) so rule_events can be explained in plain language—see 20260320140000_power_bi_dim_context.exs.

Calendar dimension and fact views

dim_calendario is built from distinct buckets of the 1-minute continuous aggregate—useful for time-intelligence style filters without scanning the full hypertable:

-- From the same migration (conceptual shape)
CREATE VIEW dim_calendario AS
SELECT DISTINCT
  bucket AS ts_bucket,
  date_trunc('day', bucket)::timestamptz AS dt,
  EXTRACT(YEAR FROM bucket)::int AS year,
  EXTRACT(MONTH FROM bucket)::int AS month,
  ...
FROM telemetry_events_1min;

Fact views project CAGG columns into a stable star join key:

CREATE VIEW fact_telemetria_agregada_1min AS
SELECT bucket AS ts_bucket,
  UPPER(SUBSTRING(fact_name FROM 1 FOR 7)) AS fbe_id,
  fact_name,
  value_float_avg AS avg_value,
  value_float_min AS min_value,
  value_float_max AS max_value
FROM telemetry_events_1min;

Parallel views exist for _1h and _1day grains. In Power BI, typical relationships are: fact → dim_equipamento_fbe on fbe_id, fact → dim_variaveis_mapeamento on fact_name, fact → dim_calendario on ts_bucket.

CAGG latency vs freshness: Continuous aggregates refresh on a policy schedule (see migrations under priv/repo/migrations for telemetry_events_1min and friends). That is ideal for import or DirectQuery models that scan fewer rows per question. When operators need charts that track the last minutes without waiting for the next rollup refresh, SmartBreweryBI hits telemetry_events directly with time_bucket—same SQL idioms, different freshness/cost trade-off.

Practical note: rule_events.regra_id is persisted as the string form of the numeric rule id (e.g. "1"). dim_regras uses keys like "r_01". For joins you may add a calculated column or a small bridge in SQL—worth standardizing in one place for your deployment.

Offline analytics: mix export.ml (with TSDB enabled) emits CSV slices of telemetry, OEE, anomalies, rules, and dimension tables—useful for Python/R notebooks or training pipelines before Part 9 on dev.to’s prediction import path.

Event facts beside telemetry

Telemetry is only one analytical slice. The same database holds oee_snapshots (availability / performance / quality components over time), anomaly_events (EMA-driven highlights aligned with the operator UI), and rule_events (which PON rule fired, with optional case_id for scenario tracking). None of these replace the star views—they complement them: a single Power BI report can place OEE line charts next to aggregated temperature trends and a table of recent rule firings, all filtered by the same time slicer on ts.

Native BI in Elixir: `SmartBreweryBI`

The Phoenix app does not require Power BI for demos. SmartBreweryBI centralizes parameterized SQL: OEE cards, telemetry trends with time_bucket, correlation pivots, synoptic averages by FBE, anomaly Pareto, and rule counts.

# SimulacoesVisuais.SmartBreweryBI (excerpt)
def default_filters do
  %{
    "window" => "24h",
    "granularity" => "1h",
    "fbe_id" => "all",
    "fact_name" => "all"
  }
end

def dashboard_data(filters) do
  filters = normalize_filters(filters)
  hours = window_to_hours(filters["window"])

  %{
    oee_cards: oee_cards(hours),
    oee_trend: oee_trend(hours, filters["granularity"]),
    telemetry_trend: telemetry_trend(filters, hours),
    cep_chart: cep_chart(filters, hours),
    correlation_points: correlation_points(filters, hours),
    synoptic_status: synoptic_status(hours),
    anomaly_pareto: anomaly_pareto(hours),
    rule_top: rule_top(hours),
    totals: totals(hours)
  }
end

The BI view mode in SmartBreweryLive (Part 6 on dev.to) loads this map when :tsdb_enabled is true—same warehouse, two consumers (LiveView vs external BI).

For near-real-time trends, the module prefers querying telemetry_events with time_bucket for selected panels so results are not delayed by CAGG refresh policies; aggregated views remain the sweet spot for heavier report workloads.

Governance: read-only database role

The migration AddPowerbiAnalyticsReadonlyRole creates powerbi_analytics: LOGIN, CONNECT to the database, USAGE on public, SELECT on tables and default privileges for future tables. The BI connector should not use the application superuser.

# Excerpt from 20250319400000_add_powerbi_analytics_readonly_role.exs
execute format('CREATE ROLE powerbi_analytics WITH LOGIN PASSWORD %L', '#{escaped}')
execute "GRANT USAGE ON SCHEMA public TO powerbi_analytics;"
execute "GRANT SELECT ON ALL TABLES IN SCHEMA public TO powerbi_analytics;"
execute "ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powerbi_analytics;"

Set POWERBI_ANALYTICS_PASSWORD before migrating in real environments.

Optional: push rows to Power BI REST

When Import/DirectQuery is not enough for a “live tile” experiment, PowerBIPushSink can enqueue rows after they are persisted, throttle HTTP calls, and POST to the Push datasets API. It is off by default (:power_bi_push → enabled: false).

# SimulacoesVisuais.SmartBrewery.PowerBIPushSink — encode_row/2 (excerpt)
defp encode_row(%{ts: ts, fact_name: fact_name, value_float: vf, value_int: vi, value_str: vs}, opts) do
  fact_str = to_string(fact_name)

  base = %{
    "ts" => datetime_to_iso8601(ts),
    "fact_name" => fact_str,
    "value_float" => vf,
    "value_int" => vi,
    "value_str" => vs
  }

  if Keyword.get(opts, :include_labels, true) do
    desc = SimulacoesVisuais.SmartBrewery.FatoDescriptions.descricao_bin(fact_str)
    Map.put(base, "descricao", desc)
  else
    base
  end
end

Operational guidance (latency vs license, DirectQuery vs push) lives in docs/power-bi-realtime.md at the repo root.

Verify the model: `mix verify.bi`

The Mix task runs SmartBreweryBI.run_query_diagnostics/0: each diagnostic query returns :ok with a small LIMIT sample, or an error you can fix before publishing reports.

# From apps/simulacoes_visuais (TSDB enabled)
mix verify.bi
# alias: mix simulacoes_visuais.verify_bi_queries

Zero rows in a sample is still success if your simulation was off—the README stresses distinguishing “query broken” from “empty time window”.

Architecture sketch

flowchart LR
  subgraph warehouse [Postgres / TimescaleDB]
    TE[telemetry_events]
    CAGG[telemetry_events_1min / 1h / 1day]
    F1[fact_telemetria_agregada_*]
    D1[dim_equipamento_fbe]
    D2[dim_variaveis_mapeamento]
    DC[dim_calendario]
    DR[dim_regras]
    OEE[oee_snapshots]
    RE[rule_events]
  end
  subgraph consumers [Consumers]
    LV[SmartBrewery BI tab]
    PBI[Power BI DirectQuery / Import]
    PUSH[Power BI Push optional]
  end
  TE --> CAGG
  CAGG --> F1
  F1 --> D1
  F1 --> D2
  F1 --> DC
  RE --> DR
  LV --> SmartBreweryBI
  SmartBreweryBI --> TE
  SmartBreweryBI --> OEE
  PBI --> F1
  PBI --> D1
  TE --> PUSH

Summary

Part 7 on dev.to wrote fast facts; this post makes them legible: conformed dimensions, CAGG-backed fact views, a read-only role, optional push, and a single Elixir module that encodes the same SQL the UI and mix verify.bi rely on. Next, we close the loop with ML on top of these exports (Part 9 on dev.to).

References and further reading

Kimball, R.; Ross, M. — The Data Warehouse Toolkit (3rd ed.) — dimensional modeling, star schema, fact/grain discipline (Wiley).
TimescaleDB — continuous aggregates as rollups — docs.timescale.com.
Microsoft Learn — Power BI REST / push datasets (see also in-repo power-bi-realtime.md).
PostgreSQL — roles, GRANT, least privilege — documentation.
In this repo — smart_brewery_bi.ex, power_bi_push_sink.ex, migrations 20250319200000_add_star_schema_dimensions_and_fact_views.exs, 20250319400000_add_powerbi_analytics_readonly_role.exs. Expanded list: Bibliography on dev.to — PON + Smart Brewery series (EN drafts) · repo draft.

Published on dev.to: BI without mystery: dimensions, facts, and consuming the data (e.g. Power BI) — tracked in docs/devto_serie_pon_smart_brewery.md.

Previous: Part 7 on dev.to — From simulation to storage: telemetry, Broadway/GenStage, and TimescaleDB · repo draft

Next: Part 9 on dev.to — ML on the digital twin: export, train pilots, and import predictions back into the app · repo draft

From simulation to storage: telemetry, Broadway/GenStage, and TimescaleDB

Matheus de Camargo Marques — Fri, 20 Mar 2026 17:06:08 +0000

If this helped you, you can support the author with a coffee on dev.to.

From simulation to storage: telemetry, Broadway/GenStage, and TimescaleDB

Part 7 of 12 — Part 6 on dev.to — Phoenix LiveView in real time: an operations UI on top of a rules engine · repo draft showed how LiveView stays responsive: batched PubSub messages and a second throttle before touching assigns. This post goes one layer deeper: what happens when those same Fato.atualizar/2 notifications must also become durable rows in PostgreSQL with the TimescaleDB extension—without blocking the rule processes or the UI.

We focus on the simulacoes_visuais Phoenix app: a GenStage producer, a Broadway pipeline with configurable batching, an asynchronous GenServer writer, and hypertables with retention. Dimensional BI and Power BI consumption are the subject of Part 8 on dev.to.

Why a separate persistence path?

The PON engine (tec0301_pon) is optimized for reactive evaluation: Registry, Fato processes, and rule notifications. Long-running Repo.insert_all/3 calls do not belong on that hot path. The Phoenix application therefore acts as a sink, using the same back-pressure vocabulary as GenStage (demand-driven stages) and Broadway (batched consumers)—see the GenStage and Broadway documentation:

Subscribe to the engine’s notification bus (via SmartBreweryFactBroadcaster).
Shape traffic with back-pressure-friendly stages (GenStage + Broadway, with a GenServer fallback batcher).
Publish merged batches to PubSub for subscribers (EMA, OEE, downstream services).
Persist only when :tsdb_enabled is true, through a queued async writer so database latency does not stall the pipeline.

Running mix run examples/smart_brewery_simulacao.exs at the monorepo root still exercises only :tec0301_pon; it does not fill telemetry_events. Only this app, with TSDB enabled, owns the warehouse path—see apps/simulacoes_visuais/README.md.

Turning the warehouse on: supervision and environment

At startup, SimulacoesVisuais.Application reads SIMULACOES_TSDB_ENABLED from the OS environment (falling back to Application.get_env(:simulacoes_visuais, :tsdb_enabled, false)). When enabled, it starts Repo plus dedicated writers before the HTTP endpoint:

# apps/simulacoes_visuais/lib/simulacoes_visuais/application.ex (excerpt)
tsdb_children =
  if tsdb_enabled do
    [
      SimulacoesVisuais.Repo,
      SimulacoesVisuais.SmartBrewery.TelemetryAsyncWriter,
      SimulacoesVisuais.SmartBrewery.OeeSnapshotWriter,
      SimulacoesVisuais.SmartBrewery.AnomalyEventWriter,
      SimulacoesVisuais.SmartBrewery.RuleEventWriter
    ]
  else
    []
  end

children =
  [
    # … PubSub, bridge, TelemetryPipeline, batchers, Monte Carlo, …
    tsdb_children,
    SimulacoesVisuaisWeb.Endpoint
  ]
  |> List.flatten()

Fact telemetry flows through TelemetryPipeline (always started) and reaches TelemetryAsyncWriter only when the flag is on. OEE, anomalies, and rule firings use separate PubSub topics and writers that also depend on Repo.

Convenience in dev: mix dev.tsdb / iex -S mix dev.tsdb aligns with the README so Postgres/TimescaleDB and Monte Carlo can run together for local demos.

GenStage producer: demand-driven ingest

SmartBreweryFactBroadcaster resolves the Broadway producer and sends GenStage.cast(producer_pid, {:event, nome_fato, novo_valor}). The producer keeps a bounded queue; when the queue is full, the oldest event is dropped—trading completeness for bounded memory under burst load.

Critically, when Broadway has already requested demand with an empty queue, handle_cast/2 drains immediately so events are not stuck until the next handle_demand:

# SimulacoesVisuais.SmartBrewery.TelemetryProducer (excerpt)
def handle_cast(
      {:event, nome_fato, valor},
      %{queue: queue, max_queue: max_queue, pending_demand: pending} = state
    ) do
  new_queue =
    if :queue.len(queue) >= max_queue do
      {{:value, _dropped}, tail} = :queue.out(queue)
      :queue.in({nome_fato, valor}, tail)
    else
      :queue.in({nome_fato, valor}, queue)
    end

  if pending > 0 do
    {events, drained_queue} = take_from_queue(new_queue, pending)
    new_pending = pending - length(events)
    {:noreply, events, %{state | queue: drained_queue, pending_demand: new_pending}}
  else
    {:noreply, [], %{state | queue: new_queue}}
  end
end

If the producer PID cannot be resolved, the broadcaster falls back to SmartBreweryTelemetryBatcher; that path must still call TelemetryAsyncWriter.cast_batch/1 on flush so TSDB does not go silent while PubSub keeps working—a parity fix called out in the batcher’s moduledoc.

Broadway: processors, batchers, and one merged batch

TelemetryPipeline uses Broadway’s batcher stage to merge many {fact, value} messages into a single map per batch (last write wins per key), then fan-out:

# SimulacoesVisuais.SmartBrewery.TelemetryPipeline (excerpt)
Broadway.start_link(__MODULE__,
  name: __MODULE__,
  producer: [
    module: {TelemetryProducer, [name: :smart_brewery_telemetry_producer]},
    transformer: {__MODULE__, :transform, []}
  ],
  processors: [default: [concurrency: processor_concurrency]],
  batchers: [
    default: [
      concurrency: batcher_concurrency,
      batch_size: batch_size,
      batch_timeout: batch_timeout
    ]
  ]
)

def handle_batch(:default, messages, _batch_info, _context) do
  if messages != [] do
    merged = messages |> merge_message_data(%{})
    list = Map.to_list(merged)

    :telemetry.execute(
      [:simulacoes_visuais, :smart_brewery_telemetry_batcher, :flush],
      %{updates_count: length(list), buffer_size_before: length(list)},
      %{}
    )

    Phoenix.PubSub.broadcast(SimulacoesVisuais.PubSub, "smart_brewery:fatos", {:batch, list})
    push_ema_numeric_loop(list)

    if Application.get_env(:simulacoes_visuais, :tsdb_enabled, false) do
      SimulacoesVisuais.SmartBrewery.TelemetryAsyncWriter.cast_batch(list)
    end
  end

  messages
end

So each batch: metrics (LiveDashboard can chart flush sizes), PubSub on smart_brewery:fatos, EMA updates for anomaly heuristics, and optionally async persist. Part 6 on dev.to’s LiveView listens on a different topic (smart_brewery:liveview_batch) fed by LiveViewEventBatcher—same simulation, two consumption speeds.

TelemetryAsyncWriter: decouple DB latency from Broadway

cast_batch/1 enqueues whole batches; a bounded queue drops the oldest batch when overloaded, keeping recent telemetry:

# SimulacoesVisuais.SmartBrewery.TelemetryAsyncWriter (excerpt)
def cast_batch(list) when is_list(list) do
  GenServer.cast(__MODULE__, {:batch, list})
end

def handle_info(:flush, %{queue: [batch | rest]} = state) do
  persist(batch)
  Process.send_after(self(), :flush, 0)
  {:noreply, %{state | queue: rest}}
end

defp persist(list) do
  rows = SimulacoesVisuais.TelemetryEvent.changesets_from_batch(list)

  if rows != [] do
    SimulacoesVisuais.Repo.insert_all(SimulacoesVisuais.TelemetryEvent, rows)
    SimulacoesVisuais.SmartBrewery.PowerBIPushSink.cast_rows(rows)
  end
end

Broadway never waits on PostgreSQL; the writer serializes inserts and can hook optional push sinks for external analytics.

Row shape: from Elixir values to columns

TelemetryEvent.changesets_from_batch/2 maps {atom_name, value} tuples into flat maps suitable for insert_all/3—numbers as value_float, booleans split across value_int / value_str, other atoms as strings—so continuous aggregates and ML exports can choose the column that fits.

# SimulacoesVisuais.TelemetryEvent (excerpt)
defp to_row(nome, valor, now) when is_number(valor) do
  %{
    ts: now,
    fact_name: Atom.to_string(nome),
    value_float: to_float(valor),
    value_int: nil,
    value_str: nil,
    inserted_at: now,
    updated_at: now
  }
end

Rule events: same bus, different writer

Rule firings are published to smart_brewery:regras (for LiveView and others). RuleEventWriter subscribes, buffers up to a max pending count, and insert_all into rule_events in chunks—again isolating database work from the rule processes.

# SimulacoesVisuais.SmartBrewery.RuleEventWriter (excerpt)
def handle_info({:regra, regra_id}, state) do
  pending = state.pending ++ [{:regra, regra_id}]
  # … trim to max_pending …
  Process.send_after(self(), :drain, 0)
  {:noreply, %{state | pending: pending, draining: true}}
end

OEE snapshots and anomaly rows follow the same pattern (dedicated modules in SmartBrewery.*).

TimescaleDB: hypertable and retention

The initial migration enables the extension, creates telemetry_events, and promotes the table to a hypertable partitioned on ts, with a 7-day retention policy on raw rows (adjustable in your deployment):

# priv/repo/migrations/20250318000000_create_telemetry_events.exs (excerpt)
execute "CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;", ""

create table(:telemetry_events, primary_key: false) do
  add :ts, :utc_datetime_usec, null: false
  add :fact_name, :string, null: false
  add :value_float, :float
  add :value_int, :integer
  add :value_str, :string
  timestamps(type: :utc_datetime_usec)
end

execute "SELECT create_hypertable('telemetry_events', 'ts', if_not_exists => true);", ""
execute "SELECT add_retention_policy('telemetry_events', INTERVAL '7 days');", ""

Later migrations add continuous aggregates (for example telemetry_events_1min, hourly and daily rollups) and compression policies on the hypertable. Those objects sit entirely in the database: the Elixir app keeps writing raw telemetry_events rows; BI queries and mix export.ml can target either the base table or the CAGGs depending on grain and export flags (--no-cagg, --no-cagg-1h-1day in the Mix task help).

Fallback batcher and TSDB parity

When the Broadway producer is unavailable, SmartBreweryTelemetryBatcher still merges updates and broadcasts {:batch, list} to smart_brewery:fatos. The flush handler mirrors the pipeline’s TSDB branch:

if Application.get_env(:simulacoes_visuais, :tsdb_enabled, false) do
  SimulacoesVisuais.SmartBrewery.TelemetryAsyncWriter.cast_batch(list)
end

Without this, operators would still see PubSub-driven UI signals while telemetry_events stayed flat—a classic split-brain symptom when debugging ingest.

End-to-end picture

flowchart TB
  subgraph engine [tec0301_pon]
    F[Fato]
  end
  subgraph ingest [simulacoes_visuais ingest]
    FB[SmartBreweryFactBroadcaster]
    P[TelemetryProducer GenStage]
    BW[TelemetryPipeline Broadway]
  end
  subgraph fanout [Fan-out]
    PS[PubSub smart_brewery:fatos]
    EMA[EMA / OEE inputs]
    W[TelemetryAsyncWriter]
  end
  subgraph db [PostgreSQL + TimescaleDB]
    T[(telemetry_events hypertable)]
    R[(rule_events etc.)]
  end
  F --> FB
  FB -->|GenStage.cast| P
  P --> BW
  BW --> PS
  BW --> EMA
  BW -->|cast_batch when tsdb_enabled| W
  W --> T
  RN[RegraNotifier] -->|PubSub regras| RW[RuleEventWriter]
  RW --> R

Operating and verifying

mix verify.tsdb — checks extension, row counts, recent timestamps, and whether the Broadway producer and writers are alive.
Headless simulation — SIMULACOES_TSDB_ENABLED=true AUTO_START_MONTE_CARLO=true mix phx.server (from the app directory) populates tables for ML export without opening the browser.
Retention — mix simulacoes_visuais.retention --days N (from the apps/simulacoes_visuais directory, TSDB enabled) replaces the hypertable retention policy without editing migrations—the default migration still installs a 7-day window.

Summary

Part 6 on dev.to optimized perception (UI). This post optimizes durability: GenStage demand, Broadway batching, bounded queues, async insert_all, and a time-series physical model. The engine stays reactive; the warehouse absorbs load on its own terms.

References and further reading

Broadway — batching, processors — hexdocs.pm/broadway.
GenStage — producers/consumers, demand — hexdocs.pm/gen_stage.
TimescaleDB — hypertables, continuous aggregates, retention — docs.timescale.com.
Telemetry — telemetry events in BEAM apps — hexdocs.pm/telemetry.
In this repo — application.ex, telemetry_pipeline.ex, telemetry_producer.ex, telemetry_async_writer.ex, smart_brewery_telemetry_batcher.ex, migration 20250318000000_create_telemetry_events.exs; docs/performance-dev.md. Expanded list: Bibliography on dev.to — PON + Smart Brewery series (EN drafts) · repo draft.

Published on dev.to: From simulation to storage: telemetry, Broadway/GenStage, and TimescaleDB — tracked in docs/devto_serie_pon_smart_brewery.md.

Previous: Part 6 on dev.to — Phoenix LiveView in real time: an operations UI on top of a rules engine · repo draft

Next: Part 8 on dev.to — BI without mystery: dimensions, facts, and consuming the data (e.g. Power BI) · repo draft

Phoenix LiveView in real time: an operations UI on top of a rules engine

Matheus de Camargo Marques — Fri, 20 Mar 2026 17:02:00 +0000

If this helped you, you can support the author with a coffee on dev.to.

Phoenix LiveView in real time: an operations UI on top of a rules engine

Part 6 of 12 — Part 5 on dev.to — Smart Brewery: a digital twin brewery as a PON lab · repo draft described the Smart Brewery digital twin: 57 Fato processes, twelve defrule modules, scripted simular/0, and the hybrid Monte Carlo loop. Operators do not introspect the Registry in iex; they need a panel. This post walks through SimulacoesVisuaisWeb.SmartBreweryLive: how Phoenix LiveView subscribes to the same changing world, why we batch updates, and how rule firings surface as log lines and visual hints—without duplicating the persistence pipeline (that is Part 7 on dev.to). LiveView’s model—server-owned state, incremental patches over a channel—is documented in the official Phoenix LiveView guides; batching here is our application-level back-pressure so high telemetry rates do not flood diff generation.

Route and scope

The SCADA-style dashboard is a standard LiveView route:

# apps/simulacoes_visuais/lib/simulacoes_visuais_web/router.ex (excerpt)
scope "/", SimulacoesVisuaisWeb do
  pipe_through :browser

  live "/smart-brewery", SmartBreweryLive, :index
  live "/smart-brewery/ml-predictions", MlPredictionsLive, :index
end

/smart-brewery is the twin console; /smart-brewery/ml-predictions previews ML-backed views (Part 9 on dev.to).

`mount`: snapshot + subscriptions + one stream

On connect, the LiveView builds a map of current fact values (best effort via Fato.obter/1), subscribes to four Phoenix PubSub topics, and initializes an event log as a LiveView stream (append-only UI list without holding an ever-growing assign).

# SimulacoesVisuaisWeb.SmartBreweryLive — mount/3 (excerpt)
def mount(_params, _session, socket) do
  _ = SimulacoesVisuais.SmartBrewery.CaseContext.new_session()

  fatos_names = SmartBrewery.fatos_names()

  fatos =
    Enum.into(fatos_names, %{}, fn nome ->
      {nome, safe_obter_fato(nome)}
    end)

  Phoenix.PubSub.subscribe(SimulacoesVisuais.PubSub, "smart_brewery:liveview_batch")
  Phoenix.PubSub.subscribe(SimulacoesVisuais.PubSub, "smart_brewery:oee")
  Phoenix.PubSub.subscribe(SimulacoesVisuais.PubSub, "smart_brewery:anomalias")
  Phoenix.PubSub.subscribe(SimulacoesVisuais.PubSub, "smart_brewery:regras")

  initial_entry = log_entry("sistema", "LiveView conectada. Aguardando notificações PON.")

  socket =
    socket
    |> assign(fatos_names: fatos_names, fatos: fatos)
    |> stream(:event_log, [initial_entry], dom_id: fn e -> "log-#{e.id}" end)

  {:ok, socket}
end

The real module chains a much larger assign/2 before stream/3 (FBE grouping, view_mode, OEE, BI filters, pending_fato_updates, sparklines, TSDB flags, and rule-flash state). The snippet shows the data path that matters here: read facts once, subscribe to four topics, seed the log stream.

smart_brewery:liveview_batch — batched fact diffs (see below).
smart_brewery:oee, smart_brewery:anomalias — operational KPIs and EMA-style anomaly hints.
smart_brewery:regras — “rule R_k fired” style messages for the operator log and FBE highlight.

Why a stream for the event log?

The operator log is a rolling tail of structured entries (id, timestamp, type, message). Appending with a normal assign would mean keeping the full list in memory and sending larger and larger diffs. LiveView streams are a good fit: each new line is stream_insert/3 (and old rows can be pruned to respect @max_log_entries), so the LiveView process does not accumulate an unbounded list in a single assign. The DOM gets stable id="log-<unique>" nodes, which also keeps diffing predictable when Monte Carlo is noisy.

The fact table, by contrast, is a keyed map (assigns.fatos) that the template can render by FBE—better for random access and numeric formatting than streaming dozens of independent row ids.

From PON notifications to PubSub: `SmartBreweryFactBroadcaster`

Tec0301Pon.PON.PubSub is the engine bus (Parts 2–5). The Phoenix app adds a bridge process that registers on every Smart Brewery fact name and forwards each {:notificacao, name, value} (or batch map) to telemetry and the UI batcher:

# SimulacoesVisuais.SmartBreweryFactBroadcaster — push_telemetry_for_fact/2 (excerpt)
defp push_telemetry_for_fact(state, nome_do_fato, novo_valor) do
  # … GenStage.cast to Broadway producer when available, else telemetry batcher fallback …

  if Map.get(state, :push_liveview_telemetry, true) do
    SimulacoesVisuais.LiveViewEventBatcher.push(nome_do_fato, novo_valor)
  end

  state
end

So: one notification fan-out from Fato still happens inside tec0301_pon; the Phoenix side adds a subscriber that never blocks the rule processes.

First coalescing layer: `LiveViewEventBatcher`

Monte Carlo ticks can touch many facts. Broadcasting one PubSub message per Fato.atualizar would multiply traffic and LiveView work. LiveViewEventBatcher is a small GenServer that merges updates into a map and flushes on a time window or max buffer size:

# SimulacoesVisuais.LiveViewEventBatcher (excerpt)
@topic "smart_brewery:liveview_batch"

def push(nome_do_fato, novo_valor) do
  GenServer.cast(__MODULE__, {:fato, nome_do_fato, novo_valor})
end

def handle_info(:flush, %{buffer: buffer} = state) do
  if buffer != %{} do
    list = Map.to_list(buffer)
    Phoenix.PubSub.broadcast(SimulacoesVisuais.PubSub, @topic, {:batch, list})
  end

  {:noreply, %{state | buffer: %{}, timer_ref: nil}}
end

Subscribers receive {:batch, [{fact, value}, ...]}—one message per window instead of dozens.

Second coalescing layer: LiveView `pending_fato_updates`

Even batched messages can arrive faster than you want to re-render heavy tables. handle_info({:batch, updates}, socket) merges into assigns.pending_fato_updates and starts a single Process.send_after to :flush_pending_fatos (interval from config, e.g. :smart_brewery_live_flush_pending_ms):

def handle_info({:batch, updates}, socket) when is_list(updates) do
  pending = Enum.into(updates, %{}, fn {k, v} -> {k, v} end)
  socket = add_pending_and_schedule_flush(socket, pending)
  {:noreply, socket}
end

defp add_pending_and_schedule_flush(socket, new_updates) when new_updates != %{} do
  pending = Map.merge(socket.assigns.pending_fato_updates, new_updates)
  socket = assign(socket, :pending_fato_updates, pending)

  if socket.assigns.flush_timer_ref do
    socket
  else
    ref = Process.send_after(self(), :flush_pending_fatos, flush_pending_ms())
    assign(socket, :flush_timer_ref, ref)
  end
end

:flush_pending_fatos applies the merged map to assigns.fatos, refreshes in-memory sparklines when TSDB is off, appends a compact log line, and clears the timer—one UI refresh per throttle window.

def handle_info(:flush_pending_fatos, socket) do
  pending = socket.assigns.pending_fato_updates

  socket =
    socket
    |> assign(:flush_timer_ref, nil)
    |> assign(:pending_fato_updates, %{})

  if pending == %{} do
    {:noreply, socket}
  else
    list = Map.to_list(pending)

    new_fatos =
      Enum.reduce(list, socket.assigns.fatos, fn {nome, valor}, acc ->
        Map.put(acc, nome, valor)
      end)

    new_spark =
      if socket.assigns.tsdb_enabled,
        do: socket.assigns.sparkline_data,
        else: sparkline_update(socket.assigns.sparkline_data, list)

    entry = log_entry("fato", "#{map_size(pending)} atualizações aplicadas")

    socket =
      socket
      |> assign(:fatos_prev, socket.assigns.fatos)
      |> assign(:fatos, new_fatos)
      |> assign(:sparkline_data, new_spark)
      |> append_event_log(entry)

    {:noreply, socket}
  end
end

The repo also merges pending_anomalia_fbes into anomalia_fbes on flush so anomaly badges align with the same frame as the fact map update—same idea: coalesce, then assign once.

When a rule fires: log cooldown and FBE flash

Part 5 on dev.to already described how rule modules notify the Phoenix side; here the LiveView only consumes {:regra, regra_id} on smart_brewery:regras. handle_info/2 resolves affected FBEs from @regras_fbe_map, calls flash_fbes_from_rule/2 for a short highlight (@regra_flash_ms), and appends to the event log at most once per cooldown per rule id (@regra_log_cooldown_ms) so chatter does not drown the operator.

def handle_info({:regra, regra_id}, socket) do
  now = DateTime.utc_now()
  last = Map.get(socket.assigns.last_regra_log_at, regra_id)
  skip? = last && DateTime.diff(now, last, :millisecond) < @regra_log_cooldown_ms

  action_fbes =
    @regras_fbe_map
    |> Map.get(regra_id, %{})
    |> Map.get(:action, [])

  socket = flash_fbes_from_rule(socket, action_fbes)

  socket =
    if skip? do
      socket
    else
      entry = log_entry("regra", "Regra disparada: #{regra_id}")

      socket
      |> append_event_log(entry)
      |> assign(:last_regra_log_at, Map.put(socket.assigns.last_regra_log_at, regra_id, now))
    end

  {:noreply, socket}
end

This ties the PON action (already executed in the rule process) to human-readable feedback—without re-running conditions in the template.

OEE and anomalies (surface only)

Separate PubSub topics keep slow-moving KPIs and exception paths from competing with the high-rate fact batch channel. handle_info({:oee_update, pct, components}, socket) when components is a map refreshes assigns.oee_percent and the component breakdown for the header badges. handle_info({:anomalia, nome_fato, _valor, _ema, _sigma}, socket) resolves the fact name to an FBE id and queues a short-lived highlight so operators notice drift without staring at raw numbers.

How OEE is computed inside SimulacoesVisuais.SmartBrewery.OEE, how EMA control limits are published, and how the same signal feeds TimescaleDB and Broadway producers are intentionally out of scope here—that is Part 7 on dev.to (pipeline and persistence).

Operator controls: scripted run and Monte Carlo

Buttons call plain handle_event callbacks:

def handle_event("run_simulacao", _params, socket) do
  lv_pid = self()
  Task.start(fn ->
    SmartBrewery.simular()
    send(lv_pid, {:simulacao_concluida})
  end)

  {:noreply, assign(socket, simulando: true)}
end

def handle_event("start_monte_carlo", _params, socket) do
  SimulacoesVisuais.SmartBreweryMonteCarlo.start_loop()
  {:noreply, assign(socket, monte_carlo_ativo: true)}
end

def handle_event("stop_monte_carlo", _params, socket) do
  SimulacoesVisuais.SmartBreweryMonteCarlo.stop_loop()
  {:noreply, assign(socket, monte_carlo_ativo: false)}
end

Long work stays out of the LiveView process; completion is a single handle_info.

View modes and DOM discipline

The LiveView defines four modes (label + user-facing description) used by the tab strip:

Key	Label	Purpose
`tabela`	Tabela	Tabular facts and live values by equipment.
`diagramas`	Diagramas	Mermaid diagrams for the process pipeline and rule graph.
`3d`	Vista 3D	3D scene with `phx-hook` for interaction; detailed static pages per FBE.
`bi`	BI (painel analítico)	Ecto-backed charts and filters—Part 8 on dev.to goes deeper on the analytical model.

Only the active view_mode subtree is rendered into the HEEx template, so switching tabs does not leave three heavy panels in the DOM paying LiveView diff cost. That matters because assigns still carries the full fact map, OEE, BI payloads, and flash state: invisible work is the enemy of smooth Monte Carlo demos on a laptop.

flowchart LR
  subgraph pon [tec0301_pon]
    Fato[Fato]
    Reg[Tec0301Pon_PubSub]
    Regra[Regra]
  end
  subgraph phx [simulacoes_visuais]
    Br[SmartBreweryFactBroadcaster]
    Bat[LiveViewEventBatcher]
    Ps[Phoenix_PubSub]
    LV[SmartBreweryLive]
  end
  Fato --> Reg
  Reg --> Br
  Br --> Bat
  Bat --> Ps
  Ps -->|"smart_brewery:liveview_batch"| LV
  Regra -->|RegraNotifier| Ps
  Ps -->|"smart_brewery:regras"| LV

What we defer

Broadway, GenStage producers, RuleEventWriter, TimescaleDB retention — Part 7 on dev.to.
BI dimensions, Power BI–style consumption — Part 8 on dev.to.
ML predictions page — Part 9 on dev.to.

Summary

LiveView is the operator-facing adapter on top of the same PON graph: bridge from engine Registry to Phoenix PubSub, batch fact fan-out, throttle assigns, stream structured log events, and surface rule/OEE/anomaly channels. The pattern scales to noisy twins—at the cost of two deliberate aggregation layers.

References and further reading

Phoenix LiveView — overview, assigns, streams — hexdocs.pm/phoenix_live_view.
Phoenix PubSub — broadcast/subscribe — hexdocs.pm/phoenix_pubsub.
Elixir Registry — duplicate-key dispatch (engine side) — HexDocs.
In this repo — smart_brewery_live.ex, smart_brewery_fact_broadcaster.ex, live_view_event_batcher.ex, router.ex. Expanded list: Bibliography on dev.to — PON + Smart Brewery series (EN drafts) · repo draft.

Published on dev.to: Phoenix LiveView in real time: an operations UI on top of a rules engine — tracked in docs/devto_serie_pon_smart_brewery.md.

Previous: Part 5 on dev.to — Smart Brewery: a digital twin brewery as a PON lab · repo draft

Next: Part 7 on dev.to — From simulation to storage: telemetry, Broadway/GenStage, and TimescaleDB · repo draft

Smart Brewery: a digital twin brewery as a PON lab

Matheus de Camargo Marques — Fri, 20 Mar 2026 16:58:56 +0000

If this helped you, you can support the author with a coffee on dev.to.

Smart Brewery: a digital twin brewery as a PON lab

Part 5 of 12 — Part 4 on dev.to — Hexagonal architecture + PON: Ports & Adapters to decouple the engine · repo draft separated ports from adapters so rule side effects stay testable. This post switches from toy examples to a single, opinionated lab: a digital twin of a brewery line implemented as a PON graph—dozens of facts, cross-equipment rules, scripted scenarios, and a hybrid simulator (physics-ish models + Monte Carlo noise) that pounds the engine the way a real plant telemetry stream would. Industry vocabulary for “twin + physical asset + lifecycle data” was popularized in part by Grieves’s digital-manufacturing framing (often cited as Digital Twin: Manufacturing Excellence Through Virtual Factory Replication, 2014); our twin is a software-only stress lab, not a certified plant model.

The code lives in the monorepo: core facts and rules under tec0301_pon, continuous simulation under the simulacoes_visuais Phoenix app.

What we are simulating

The twin groups 57 facts into eleven functional block elements (FBEs)—mill, mash, filter, boil, heat exchanger, two fermenters, packaging, CIP, AMR fleet, and a smart grid slice. Twelve rules (R_01–R_12) encode operational and safety logic: filtration optimization, packaging interlocks, peak-load shifting, ISO 10816-3 mill protection, NR-13 boiler limits, ISA-88 mash–filter coordination, AMR battery policy, and grid resilience—see the module documentation in Tec0301Pon.Examples.SmartBrewery and the full attribute/rule catalog in docs/smart-brewery-fatos-regras.md (repository root docs/).

FBE	Name (concept)	Fact count
01	Mill / grist	5
02	Mash tun	6
03	Lauter / filter	5
04	Boil kettle	5
05	Heat exchanger	4
06	Fermenter A	7
07	Fermenter B	6
08	Bottling line	6
09	CIP	5
10	AMR fleet	5
11	Smart grid	4

Total: 57 facts, all atoms backed by Tec0301Pon.PON.Fato processes, with the usual Registry fan-out from Parts 2–3.

Bootstrapping the graph

Tec0301Pon.Examples.SmartBrewery.start_link/0 starts every fact from the initial-value list, then starts each generated rule module (R_01 … R_12):

# lib/tec0301_pon/examples/smart_brewery.ex (conceptual excerpt)
def start_link do
  for {nome, valor} <- @fatos_iniciais do
    Fato.start_link(nome, valor)
  end

  Tec0301Pon.Examples.SmartBrewery.Regras.RegraOtimizacaoFiltracao.start_link()
  Tec0301Pon.Examples.SmartBrewery.Regras.RegraIntertravamentoEnvase.start_link()
  Tec0301Pon.Examples.SmartBrewery.Regras.RegraSmartGridLoadBalancing.start_link()
  # ... remaining rules R_04 – R_12
  {:ok, self()}
end

In the Phoenix app, a bridge supervisor typically hosts this graph so LiveView and the Monte Carlo loop see running processes (SimulacoesVisuais.Application children—Part 6 on dev.to focuses on the UI).

Rules in the wild: R_01 and R_04

Rules use Tec0301Pon.PON.Builder (defrule). R_01 tightens filtration when differential pressure is high, clarity is poor, and pump speed is aggressive:

# lib/tec0301_pon/examples/smart_brewery_regras.ex (excerpt — R_01)
defrule(RegraOtimizacaoFiltracao,
  watch: [:fbe_03_diff_pressure, :fbe_03_wort_clarity, :fbe_03_pump_speed],
  when:
    memoria[:fbe_03_diff_pressure] > 150 and memoria[:fbe_03_wort_clarity] < 20 and
      memoria[:fbe_03_pump_speed] > 50,
  do:
    (
      Tec0301Pon.Examples.SmartBrewery.FBE_03.reduce_pump_10pct()
      Tec0301Pon.Examples.SmartBrewery.FBE_03.lower_rake_position()
      Tec0301Pon.Examples.SmartBrewery.RegraNotifier.notify(:r_01)
    )
)

R_04 protects the mill using edge_triggered: true so the action does not re-fire on every notification while the condition stays true (Part 3 on dev.to):

# lib/tec0301_pon/examples/smart_brewery_regras.ex (excerpt — R_04)
defrule(RegraProtecaoMoinho,
  watch: [
    :fbe_01_motor_temp,
    :fbe_01_vibration_level,
    :fbe_01_hopper_level,
    :fbe_01_motor_rpm,
    :fbe_01_feed_valve_state
  ],
  when:
    (memoria[:fbe_01_vibration_level] != nil and memoria[:fbe_01_vibration_level] > 80) or
      (memoria[:fbe_01_motor_temp] != nil and memoria[:fbe_01_motor_temp] > 70) or
      (memoria[:fbe_01_hopper_level] != nil and memoria[:fbe_01_motor_rpm] != nil and
         memoria[:fbe_01_hopper_level] < 15 and memoria[:fbe_01_motor_rpm] > 0),
  edge_triggered: true,
  do:
    (
      Tec0301Pon.Examples.SmartBrewery.FBE_01.reduce_motor_rpm()

      if memoria[:fbe_01_vibration_level] != nil and memoria[:fbe_01_vibration_level] > 95,
        do: Tec0301Pon.Examples.SmartBrewery.FBE_01.close_feed_valve()

      Tec0301Pon.Examples.SmartBrewery.RegraNotifier.notify(:r_04)
    )
)

FBE_XX modules update facts to represent equipment response; RegraNotifier feeds observability (PubSub / persistence hooks in the full app).

Scripted walkthrough: `simular/0`

For demos and tests, SmartBrewery.simular/0 drives a narrative sequence of Fato.atualizar/2 calls to trigger R_01, then R_02, then R_03:

# lib/tec0301_pon/examples/smart_brewery.ex (excerpt)
def simular do
  Fato.atualizar(:fbe_03_wort_clarity, 15)
  Fato.atualizar(:fbe_03_pump_speed, 60)
  # ...
  Fato.atualizar(:fbe_03_diff_pressure, 152)
  # ...
  Fato.atualizar(:fbe_08_capper_jam_sens, true)
  # ...
  Fato.atualizar(:fbe_09_cip_pump_state, :on)
  Fato.atualizar(:fbe_11_grid_power_cost, 180)
end

This is the inbound adapter story from Part 4 in concrete form: something external (here, a script) pushes world state; rules react.

Continuous stress: hybrid Monte Carlo

Long-running exploration uses SimulacoesVisuais.SmartBreweryMonteCarlo—a GenServer that schedules ticks. Each tick runs dedicated models for key subsystems, then applies stochastic updates to a subset of remaining facts according to a schema ({:range, min, max}, {:enum, [...]}, :bool). Some continuous variables use a random walk (Gaussian step) instead of i.i.d. uniforms.

# apps/simulacoes_visuais/lib/.../smart_brewery_monte_carlo.ex (excerpt)
def run_tick_pure(state) do
  FBE03Darcy.tick()
  FBE06Fermentation.tick()
  FBE07Fermentation.tick()
  FBE08Markov.tick()
  FBE10Markov.tick()
  FBE11SmartGrid.tick()
  state = update_fbe03_cholesky(state)
  # pick N fact names from @mc_fact_names, then:
  apply_mc_chosen_updates(chosen)
  state
end

defp apply_mc_chosen_updates([nome | rest]) do
  valor = next_value(nome)
  if valor != nil do
    Fato.atualizar(nome, valor)
  end
  apply_mc_chosen_updates(rest)
end

Scheduling:

def handle_info(:tick, state) do
  new_state = run_tick_pure(state)
  Process.send_after(self(), :tick, state.interval_ms)
  {:noreply, new_state}
end

Facts owned by the dedicated models are excluded from the naive Monte Carlo list (@excluded_from_mc) so two writers do not fight. The LiveView Start / Stop Monte Carlo buttons call SmartBreweryMonteCarlo.start_loop/0 and stop_loop/0 (Part 6 on dev.to).

flowchart LR
  subgraph sim [Simulation_tick]
    MC[MonteCarlo_subset]
    M03[FBE03Darcy]
    M06[FBE06Fermentation]
    M08[FBE08Markov]
  end
  Fato[Tec0301Pon_Fato]
  Reg[Registry_PubSub]
  Regra[Regra_processes]
  FBE[FBE_side_effects]
  sim --> Fato
  Fato --> Reg
  Reg --> Regra
  Regra --> FBE
  Regra --> Notif[RegraNotifier]
  subgraph later [Later_posts]
    LV[LiveView]
    TSDB[TSDB_writers]
  end
  Notif -.-> LV
  Notif -.-> TSDB

Telemetry, TSDB, and the Phoenix app (teaser)

With :tsdb_enabled, the simulacoes_visuais app starts Ecto + async writers (e.g. rule events) fed from the operational pipeline—Part 7 on dev.to covers Broadway, batching, and TimescaleDB in depth. Part 6 on dev.to covers LiveView (SmartBreweryLive), streams, and operator UX.

Heads-up: a root mix run examples/smart_brewery_simulacao.exs path may start only :tec0301_pon and not persist to telemetry_events; full TSDB behavior requires the Phoenix app with the right config—see apps/simulacoes_visuais/README.md.

Summary

The Smart Brewery twin is a deliberately large PON graph: many facts, cross-FBE rules, FBE modules that close the loop on state, plus two simulation modes—a scripted simular/0 for storytelling and a hybrid Monte Carlo loop for sustained load. It is the same engine you built in Parts 1–4, now exercised like a product.

References and further reading

Grieves, M. — Digital Twin: Manufacturing Excellence Through Virtual Factory Replication (2014 white paper; search institutional copies) — vocabulary for twins in manufacturing.
Simão et al. (2013) — NOP / notification-oriented control — SCIRP (conceptual link to PON-style updates).
In this repo — docs/smart-brewery-fatos-regras.md; lib/tec0301_pon/examples/smart_brewery.ex, smart_brewery_regras.ex, smart_brewery_fbe.ex; apps/simulacoes_visuais/lib/simulacoes_visuais/smart_brewery_monte_carlo.ex; SimulacoesVisuaisWeb.SmartBreweryLive. Expanded list: Bibliography on dev.to — PON + Smart Brewery series (EN drafts) · repo draft.

Published on dev.to: Smart Brewery: a digital twin brewery as a PON lab — tracked in docs/devto_serie_pon_smart_brewery.md.

Previous: Part 4 on dev.to — Hexagonal architecture + PON: Ports & Adapters to decouple the engine · repo draft

Next: Part 6 on dev.to — Phoenix LiveView in real time: an operations UI on top of a rules engine · repo draft.

Hexagonal architecture + PON: Ports & Adapters to decouple the engine

Matheus de Camargo Marques — Fri, 20 Mar 2026 16:54:03 +0000

If this helped you, you can support the author with a coffee on dev.to.

Hexagonal architecture + PON: Ports & Adapters to decouple the engine

Part 4 of 12 — Part 3 on dev.to — A metaprogrammed DSL: defrule and defpremissa with less PON boilerplate · repo draft added defrule and defpremissa so you declare watch / when / do instead of hand-written Regra callbacks. This post draws a boundary between that reactive core and the messy world: databases, HTTP, MQTT, GPIO, SMS APIs. In Elixir, Ports & Adapters map cleanly to behaviours (@callback) and implementing modules—the same pattern this repository already uses under lib/tec0301_pon/ports/ and lib/tec0301_pon/adapters/.

Why hexagonal thinking matters for rules

A rule’s do: block is the natural place for side effects: sound an alarm, open a valve, enqueue a job. If you import HTTPoison, GPIO, or an Ecto repo inside the rule module, the PON graph becomes hard to test without the network, hardware, or database. Worse, you cannot swap implementations between dev, CI, and production without editing rule code.

Hexagonal architecture—as Cockburn described in his ports and adapters model (original article; see also Fowler’s summary)—keeps the application core ignorant of those technologies. The core defines ports—interfaces for what it needs. Adapters sit outside and satisfy those interfaces with concrete I/O.

For PON specifically:

Inbound (driving): anything that writes facts—sensors, Phoenix controllers, LiveView events, cron jobs—eventually calls Fato.atualizar/2 (or your own thin wrapper). Part 6 on dev.to revisits UI; here we only name the direction.
Outbound (driven): what rules trigger when conditions hold—notifications, actuators, persistence. This repo’s examples focus on outbound behaviours.

flowchart TB
  subgraph inbound [Inbound_adapters]
    Sensor[Sensor_or_UI]
  end
  subgraph core [PON_core]
    Fato[Fato]
    Regra[Regra]
  end
  subgraph port_layer [Port_behaviour]
    PortMod[Ports_PredioAtuadores]
  end
  subgraph outbound [Outbound_adapters]
    IO[Adapters_PredioIO]
    RealHW[Real_HVAC_API]
  end
  Sensor -->|"Fato.atualizar"| Fato
  Fato --> Regra
  Regra -->|"calls contract"| PortMod
  PortMod -.->|"implements"| IO
  IO --> RealHW

Ports in Elixir: `@callback` modules

A port here is not the OTP Port for external OS processes—it is a behaviour contract. Example from the library:

# lib/tec0301_pon/ports/predio_atuadores.ex (excerpt)
defmodule Tec0301Pon.Ports.PredioAtuadores do
  @moduledoc """
  Porta (Behaviour) para atuadores do prédio inteligente: iluminação, HVAC, porta de segurança.
  """
  @callback ligar_luz() :: :ok
  @callback desligar_luz() :: :ok
  @callback ligar_ar() :: :ok
  @callback ventilar() :: :ok
  @callback trancar_porta() :: :ok
end

Another outbound port is Tec0301Pon.Ports.Alarme with @callback disparar(motivo :: String.t()) :: :ok.

The core depends only on these function signatures, not on SMS gateways or relay boards.

Adapters: `@behaviour` + `@impl true`

An adapter implements the port. The PoC ships IO-based simulations you can replace with real clients later:

# lib/tec0301_pon/adapters/predio_io.ex (excerpt)
defmodule Tec0301Pon.Adapters.PredioIO do
  @moduledoc """
  Adaptador de saída para prédio inteligente (iluminação, HVAC, porta) — simulação com IO.
  """
  @behaviour Tec0301Pon.Ports.PredioAtuadores

  @impl true
  def ligar_luz do
    IO.puts("[Prédio] Iluminação: Luz LIGADA.")
    :ok
  end

  # ... desligar_luz, ligar_ar, ventilar, trancar_porta
end

# lib/tec0301_pon/adapters/alarme_io.ex (excerpt)
defmodule Tec0301Pon.Adapters.AlarmeIO do
  @behaviour Tec0301Pon.Ports.Alarme

  @impl true
  def disparar(motivo) do
    IO.puts("[API] SMS/Push: ALARM - #{motivo}")
    :ok
  end
end

The test suite smoke-checks the adapter against the port (examples_coverage_test.exs asserts each PredioIO callback returns :ok).

Rules that call adapters: `PredioInteligente`

Tec0301Pon.Examples.PredioInteligente.Regras uses the DSL and aliases concrete adapter modules:

# lib/tec0301_pon/examples/predio_inteligente_regras.ex (excerpt)
use Tec0301Pon.PON.Builder
alias Tec0301Pon.Adapters.AlarmeIO
alias Tec0301Pon.Adapters.PredioIO

defrule(RegraEmergencia,
  watch: [:predio_alarme_incendio],
  when: (memoria[:predio_alarme_incendio] || false) == true,
  do:
    (
      AlarmeIO.disparar("Alarme de incêndio — modo emergência ativado.")
      Tec0301Pon.PON.Fato.atualizar(:predio_modo_emergencia, true)
    )
)

defrule(RegraVentilarCO2,
  watch: [:predio_co2_alto, :predio_modo_emergencia],
  when:
    (memoria[:predio_co2_alto] || false) == true and
      (memoria[:predio_modo_emergencia] || true) == false,
  do:
    (
      PredioIO.ventilar()
    )
)

This is pragmatic: the rules are readable, and the behaviour still documents the contract PredioIO must honor. For strict hexagonal purity, the rule would call only a port-facing module resolved at runtime (next section).

Production-shaped indirection: config or facade

To swap adapters in tests or per environment without changing rule source, introduce a thin facade that delegates to a module from config:

# Example pattern (not required by the PoC — add in your app)
defmodule MyApp.Building do
  @predio_mod Application.compile_env(:my_app, :predio_atuadores, Tec0301Pon.Adapters.PredioIO)

  def ventilar, do: @predio_mod.ventilar()
  def trancar_porta, do: @predio_mod.trancar_porta()
  # ...
end

Runtime (non-compile) variant:

defmodule MyApp.Building do
  def predio_mod do
    Application.get_env(:my_app, :predio_atuadores, Tec0301Pon.Adapters.PredioIO)
  end

  def ventilar, do: predio_mod().ventilar()
end

In config/test.exs:

config :my_app, :predio_atuadores, MyApp.Test.PredioStub

Rules then call MyApp.Building.ventilar() instead of PredioIO.ventilar(). The port (PredioAtuadores) is what you @behaviour in both PredioIO and PredioStub.

Stub adapter for tests

defmodule MyApp.Test.PredioStub do
  @behaviour Tec0301Pon.Ports.PredioAtuadores

  @impl true
  def ligar_luz, do: record(:ligar_luz)
  @impl true
  def desligar_luz, do: record(:desligar_luz)
  @impl true
  def ligar_ar, do: record(:ligar_ar)
  @impl true
  def ventilar, do: record(:ventilar)
  @impl true
  def trancar_porta, do: record(:trancar_porta)

  defp record(op) do
    # Rule actions run in the Regra process — send to a named test process, not self().
    if pid = Process.whereis(:predio_stub_sink), do: send(pid, {:predio_stub, op})
    :ok
  end
end

Register :predio_stub_sink as the test process (Process.register(self(), :predio_stub_sink) in setup), fire the graph, then assert_receive {:predio_stub, :ventilar}. Alternatively assert only on fact values and keep the stub as :ok no-ops.

Inbound in three lines

An inbound adapter translates an external event into a fact change:

# Conceptual sensor → PON
defmodule MyApp.Adapters.MqttTempSensor do
  def on_message(payload) do
    Tec0301Pon.PON.Fato.atualizar(:ambient_temp, payload.temperature_c)
  end
end

The rule graph does not know MQTT exists; it only reacts once :ambient_temp updates.

What we defer

Smart Brewery domain, simulation volume, and telemetry—Part 5 on dev.to onward.
LiveView as an inbound adapter—Part 6 on dev.to.
Full dependency injection framework—behaviours + Application.get_env/3 are enough for many Elixir apps.

Summary

Layer	In this repo	Role
Port	`Tec0301Pon.Ports.*`	`@callback` contract for outbound effects
Adapter	`Tec0301Pon.Adapters.*`	`@behaviour` implementation (IO, HTTP, etc.)
Rules	`Examples.PredioInteligente.Regras`	DSL; today alias adapters directly; facades + config tighten the hexagon

Hexagonal boundaries do not replace PON—they wrap it: facts and rules stay reactive; adapters keep I/O at the edges.

References and further reading

Cockburn, A. — Hexagonal architecture — alistair.cockburn.us.
Fowler, M. — Hexagonal architecture (bliki) — martinfowler.com.
Elixir — @callback, @behaviour — HexDocs Module.
In this repo — lib/tec0301_pon/ports/, lib/tec0301_pon/adapters/, lib/tec0301_pon/examples/predio_inteligente_regras.ex; Tec0301Pon @moduledoc in lib/tec0301_pon.ex. Expanded list: Bibliography on dev.to — PON + Smart Brewery series (EN drafts) · repo draft.

Published on dev.to: Hexagonal architecture + PON: Ports & Adapters to decouple the engine — tracked in docs/devto_serie_pon_smart_brewery.md.

Previous: Part 3 on dev.to — A metaprogrammed DSL: defrule and defpremissa with less PON boilerplate · repo draft

Next: Part 5 on dev.to — Smart Brewery: a digital twin brewery as a PON lab · repo draft.

A metaprogrammed DSL: defrule and defpremissa with less PON boilerplate

Matheus de Camargo Marques — Fri, 20 Mar 2026 16:47:59 +0000

If this helped you, you can support the author with a coffee on dev.to.

A metaprogrammed DSL: `defrule` and `defpremissa` with less PON boilerplate

Part 3 of 12 — Part 1 on dev.to — PON in Elixir: why the BEAM fits reactive rules · repo draft introduced PON and the raw Fato / Regra API. Part 2 on dev.to — From whiteboard to code: mapping Facts, Rules, and Premises to OTP processes · repo draft traced notifications through Registry and GenServer processes. This post adds the ergonomic layer: macros in Tec0301Pon.PON.Builder that generate small modules so you write watch / when / do instead of hand-rolling avaliar/1, executar/1, and start_link/0 every time.

Why macros here?

In Elixir, a rule backed by Tec0301Pon.PON.Regra in module mode needs:

A function that implements when the rule should fire (avaliar/1 over a memoria map).
A function that performs the action (executar/1).
A start_link/0 that calls Regra.start_link(watched_facts, __MODULE__, ...).

A premise backed by Tec0301Pon.PON.Premissa needs a condition and a start_link/0 that wires Premissa.start_link/4.

That is repetitive and easy to get subtly wrong. The Builder macros expand at compile time into nested modules under your namespace (e.g. MyApp.Rules.Cooling inside MyApp.Rules), each with the right callbacks and start_link/0. At runtime you only start processes—no macro cost on the hot path. That split—compile-time code generation vs runtime message handling—is standard Elixir metaprogramming (quote / unquote, __CALLER__); see the official Macro module and McCord’s Metaprogramming Elixir (Pragmatic) for the wider pattern language.

defmodule MyApp.Rules do
  use Tec0301Pon.PON.Builder
  # defpremissa ... and defrule ... live here
end

use Tec0301Pon.PON.Builder only imports the macro definitions; the heavy lifting is in defrule, defpremissa, and friends.

What `defrule` expands to (conceptually)

Each defrule Name, watch: [...], when: ..., do: ... becomes a module MyApp.Rules.Name (name parts are taken from the macro’s identifier and concatenated with the caller module).

Roughly, the generated module contains:

avaliar(memoria) — body of your when: clause (injected AST).
executar(memoria) — your do: block, or a loop of Task.start(mod, fun, args) when you use instigations.
start_link/0 — Tec0301Pon.PON.Regra.start_link(watched_facts, __MODULE__, opts) so the rule process uses this module’s avaliar/1 and executar/1.

That last point matters: because Regra holds an atom referring to your generated module, you can upgrade the module in a release and change rule logic while facts keep running—hot code swapping is a BEAM feature the DSL is aligned with (operational details stay out of this post).

Expansion sketch (not the real quoted AST)

If you wrote defrule Example, watch: [:a], when: memoria[:a] == 1, do: :ok inside MyApp.Rules, compile-time expansion conceptually yields something like:

# Conceptual — actual code is built with Macro.quote in Builder
defmodule MyApp.Rules.Example do
  def avaliar(memoria), do: memoria[:a] == 1
  def executar(memoria), do: :ok

  def start_link do
    Tec0301Pon.PON.Regra.start_link([:a], __MODULE__, edge_triggered: false)
  end
end

defpremissa is analogous: a nested module with condicao/1 and start_link/0 calling Premissa.start_link/4. Inspect expanded code in a scratch module with Macro.expand if you enjoy reading quote output.

Options worth knowing

edge_triggered: true — passed through to Regra.start_link/3; the action runs only on false → true transitions of the condition (see Part 2 / Regra docs).
do: [instigations: [{Mod, :fun, [args]}, ...]] — instead of an arbitrary block, spawn Tasks to call MFA tuples when the rule fires. Useful to fan out work or call adapters without blocking the rule process.
when: as a string — a separate macro clause compiles to Code.eval_string/2 over memoria: memoria. That exists for flexibility (e.g. config-driven rules), but prefer the normal AST when: in application code: eval’d strings are harder to test, refactor, and secure.

Example with a normal when: and do: block (brewery-flavored, same spirit as the project README):

defmodule MyApp.Rules do
  use Tec0301Pon.PON.Builder

  defpremissa HighAmbient,
    watch: [:ambient_temp],
    when: (memoria[:ambient_temp] || 0) > 30,
    derive: :high_temp,
    criar_fato: true

  defrule StartCooling,
    watch: [:high_temp, :compressor_state],
    when: memoria[:high_temp] == true and memoria[:compressor_state] == :off,
    do: MyApp.Actuators.start_compressor()
end

Instigations

Pattern borrowed from the test suite (Tec0301Pon.PON.BuilderTest.RegrasInstigations):

defmodule MyApp.Rules do
  use Tec0301Pon.PON.Builder

  defrule NotifyOps,
    watch: [:alarm_level],
    when: memoria[:alarm_level] == :critical,
    do: [instigations: [{MyApp.Notifications, :page_on_call, []}]]
end

When the condition holds, the generated executar/1 starts a Task per {mod, fun, args}.

String `when:` (discouraged for most apps)

defrule ThresholdFromConfig,
  watch: [:sensor_x],
  when: "memoria[:sensor_x] > 10",
  do: :ok

This generates avaliar/1 that evaluates the string at runtime. Use sparingly; never feed untrusted input into that string.

What `defpremissa` expands to

defpremissa Name, watch: [...], when: ..., derive: :fact [, criar_fato: true] generates a submodule whose start_link/0 calls:

Premissa.start_link(derive_fact, watch_list, &condicao/1, criar_fato_derivado: ...)

with condicao/1 equal to your when: body.

watch — source fact atoms (one or two in the underlying Premissa API).
derive — the derived fact name that other rules can watch.
criar_fato: true — if the derived fact process does not exist yet, start it with initial value false before subscribing.

Behavior matches Part 2: the premise process registers on the Registry keys for source facts and only calls Fato.atualizar/2 on the derived fact when the boolean result changes, avoiding notification spam.

Bonus: `defcondicao` — aggregate premises

When several boolean facts must be combined (AND/OR) before a single downstream rule fires, defcondicao generates a submodule whose start_link/0 delegates to Tec0301Pon.PON.Condicao with either:

merge: :all — every watched value must be strictly true (AND).
merge: :any — at least one true (OR).
when: expr — custom combination over memoria.

defmodule MyApp.Rules do
  use Tec0301Pon.PON.Builder

  defcondicao AlarmArmed,
    watch: [:high_temp, :low_pressure],
    merge: :all,
    derive: :alarm_ready,
    criar_fato: true

  defrule SoundAlarm,
    watch: [:alarm_ready],
    when: memoria[:alarm_ready] == true,
    do: MyApp.Actuators.sound_alarm()
end

You cannot pass both merge: and when:; the macro raises at compile time.

Bootstrap order (still your responsibility)

The DSL does not start your supervision tree for you. A reliable order is:

Application.ensure_all_started(:tec0301_pon) (or include the app in your release) so ETS and Tec0301Pon.PON.PubSub exist.
Start every raw fact the graph needs with Fato.start_link(name, initial_value) (source sensors and any derived facts unless criar_fato: true creates them).
Start premises and conditions (MyApp.Rules.HighAmbient.start_link/0, MyApp.Rules.AlarmArmed.start_link/0, …).
Start rules (MyApp.Rules.StartCooling.start_link/0, …).

Integration tests in the repo follow this pattern under Tec0301Pon.PonCase with optional Tec0301Pon.PON.Service registration for global statistics.

Minimal end-to-end sketch

Application.ensure_all_started(:tec0301_pon)

alias Tec0301Pon.PON.Fato
alias MyApp.Rules

{:ok, _} = Fato.start_link(:ambient_temp, 22)
{:ok, _} = Fato.start_link(:compressor_state, :off)

{:ok, _} = Rules.HighAmbient.start_link()
{:ok, _} = Rules.StartCooling.start_link()

Fato.atualizar(:ambient_temp, 35)
# HighAmbient may set :high_temp; StartCooling may run if compressor is :off

Adjust module names to match your defrule / defpremissa identifiers.

Side effects and Part 4

Examples above call MyApp.Actuators or MyApp.Notifications directly inside do: or instigations. That is clear for a blog post, but in a larger system you will want ports and adapters so the PON core stays testable and swappable. Part 4 on dev.to covers hexagonal boundaries around the engine; the macros stay the same—you only change what executar/1 and instigations call.

Summary

Macro	Generates	Runtime process
`defrule`	`avaliar/1`, `executar/1`, `start_link/0`	`Regra` (module mode)
`defpremissa`	`condicao/1`, `start_link/0`	`Premissa`
`defcondicao`	`start_link/0` (+ optional `combine/1`)	`Condicao`

Tec0301Pon.PON.Builder is the compile-time bridge between declarative PON graphs and the OTP mechanics from Part 2. Prefer AST when: clauses, understand edge_triggered and instigations, and start facts before premises and rules.

References and further reading

Elixir Macro — quote, unquote, hygiene — HexDocs.
McCord, C. — Metaprogramming Elixir (Pragmatic Bookshelf) — DSLs and use/__using__ patterns.
Elixir Kernel.SpecialForms — defmacro — HexDocs.
In this repo — lib/tec0301_pon/pon/builder.ex; mix docs for Tec0301Pon.PON.Builder, Regra, Premissa, Condicao. Expanded list: Bibliography on dev.to — PON + Smart Brewery series (EN drafts) · repo draft.

Published on dev.to: A metaprogrammed DSL: defrule and defpremissa with less PON boilerplate — tracked in docs/devto_serie_pon_smart_brewery.md.

Previous: Part 2 on dev.to — From whiteboard to code: mapping Facts, Rules, and Premises to OTP processes · repo draft

Next: Part 4 on dev.to — Hexagonal architecture + PON: Ports & Adapters to decouple the engine · repo draft.

From whiteboard to code: mapping Facts, Rules, and Premises to OTP processes

Matheus de Camargo Marques — Fri, 20 Mar 2026 16:39:52 +0000

If this helped you, you can support the author with a coffee on dev.to.

From whiteboard to code: mapping Facts, Rules, and Premises to OTP processes

Part 2 of 12 — In Part 1 on dev.to — PON in Elixir: why the BEAM fits reactive rules · repo draft we motivated the Notification-Oriented Paradigm (PON) and showed the public Fato / Regra API. This post opens the engine room: how updates become messages, and how rules and premises attach to fact names using OTP building blocks—without yet using the metaprogrammed DSL (defrule / defpremissa), which is Part 3 on dev.to.

The whiteboard model

On the board you draw facts (ovals), rules (boxes that watch facts), and sometimes premises (derived facts). The arrow is always the same: when this fact changes, wake up everything that cares.

On the BEAM, a practical mapping is (the OTP design space—processes under supervision exchanging messages—is laid out in Armstrong’s thesis and in Cesarini & Thompson’s Programming Erlang; here we use only a thin slice):

Concept	In `tec0301_pon`
Fact	A named `GenServer` (`Tec0301Pon.PON.Fato`) — process name = fact name (atom).
Bus	A duplicate-key `Registry` named `Tec0301Pon.PON.PubSub` — registry key = fact atom; many processes can register under the same key.
Rule	A `GenServer` (`Tec0301Pon.PON.Regra`) that registers under each watched fact key and handles `{:notificacao, ...}` (and batch variants).
Premise	Another subscriber process (`Tec0301Pon.PON.Premissa`) that writes a derived fact when a boolean condition changes.

flowchart LR
  subgraph bus [Registry_PubSub]
    topicA[fact_key_A]
  end
  FatoA[Fato_GenServer_A] -->|"dispatch on key A"| topicA
  topicA -->|send| Regra[Regra_process]
  topicA -->|send| Premissa[Premissa_process]
  Premissa -->|Fato.atualizar| FatoB[Fato_derived]

Application startup: ETS + Registry

When the tec0301_pon application starts, it ensures a shared ETS table for fast fact reads and starts the Registry that acts as the notification bus.

# lib/tec0301_pon/application.ex (excerpt)
def start(_type, _args) do
  :ok = Tec0301Pon.PON.Fato.ensure_ets!()

  children = [
    {Registry,
     keys: :duplicate, name: Tec0301Pon.PON.PubSub, partitions: System.schedulers_online()}
  ]

  opts = [strategy: :one_for_one, name: Tec0301Pon.Supervisor]
  Supervisor.start_link(children, opts)
end

Why :duplicate keys? Under a unique-key registry, only one process could register per fact name. Here, many rules (and premises) may watch :temperature. Duplicate keys mean “topic” semantics: one key, N subscribers.

Partitions spread registry work across schedulers—useful when many processes register and dispatch runs hot (later posts revisit performance).

Fact process: store, compare, dispatch

Each fact is a GenServer registered both as an OTP name (name: nome_do_fato) and as the topic for subscribers. When you call Fato.atualizar(fact, value), the fact process compares the new value to the old with ===. If nothing changed, it does not call Registry.dispatch—a cheap guard against useless fan-out (Part 10 on dev.to goes deeper on storms and batching).

When the value does change, it updates ETS and dispatches to every {pid, _value} registered under that fact’s atom:

# lib/tec0301_pon/pon/fato.ex — handle_cast({:atualizar, novo_valor}, estado) (excerpt)
Registry.dispatch(Tec0301Pon.PON.PubSub, estado.nome, fn inscritos ->
  for {pid, _} <- inscritos, do: send(pid, {:notificacao, estado.nome, novo_valor})
end)

The message shape {:notificacao, nome_fato, novo_valor} is the contract every subscriber implements in handle_info/2.

Batch path (preview): Fato.atualizar_lote/1 funnels through Tec0301Pon.PON.Fanout and can deliver {:notificacoes_lote, %{fato => valor, ...}} so a rule gets one message per burst instead of dozens. Rules handle both shapes.

Rule process: register, seed memory, evaluate

When a rule starts, it subscribes to each watched fact and seeds its local memoria map by reading current values (Fato.obter/1), so the first evaluation sees a consistent snapshot.

# lib/tec0301_pon/pon/regra.ex — init/1 (excerpt)
Enum.each(estado.fatos, fn fato ->
  Registry.register(Tec0301Pon.PON.PubSub, fato, [])
end)

memoria_inicial =
  Enum.reduce(estado.fatos, %{}, fn fato, acc ->
    valor = Tec0301Pon.PON.Fato.obter(fato)
    Map.put(acc, fato, valor)
  end)

{:ok, %{estado | memoria: memoria_inicial}}

On each {:notificacao, ...}, the rule updates memoria, drains additional notifications already in the mailbox (reducing duplicate work in bursts), then runs the condition and possibly the action. A second clause handles {:notificacoes_lote, updates} by merging only the keys the rule watches.

Premise process: same bus, different side effect

Tec0301Pon.PON.Premissa is “rule-like” in that it registers on source facts and receives the same notifications. Instead of firing an arbitrary action, it evaluates condicao_fn.(memoria) and, when the boolean changes, calls Fato.atualizar(nome_fato_derivado, novo_resultado). That derived fact becomes another topic: downstream rules only watch the derived name.

Options include criar_fato_derivado: true, which starts the derived fact as false if it did not exist—handy in scripts and tests.

alias Tec0301Pon.PON.{Fato, Premissa}

{:ok, _} = Fato.start_link(:sensor_temp, 18)

{:ok, _} =
  Premissa.start_link(
    :temp_high,
    [:sensor_temp],
    fn mem -> (mem[:sensor_temp] || 0) > 25 end,
    criar_fato_derivado: true
  )

Fato.atualizar(:sensor_temp, 30)
# Premissa updates :temp_high to true; subscribers to :temp_high get notified.

Start source facts before premises and rules that depend on them; order matters because obter/1 and initial premise evaluation run at init time.

Runnable sketch in `iex`

With the library as a dependency (or path: in mix.exs), after Application.ensure_all_started(:tec0301_pon):

alias Tec0301Pon.PON.{Fato, Regra}

{:ok, _} = Fato.start_link(:counter, 0)

condition = fn mem -> mem[:counter] >= 3 end
action = fn _mem -> IO.puts("counter reached 3") end

{:ok, rule_pid} = Regra.start_link([:counter], condition, action)

Fato.atualizar(:counter, 1)
Fato.atualizar(:counter, 3)
# When the condition evaluates to true after an update, the action runs.
# Use Regra.start_link(..., modulo, edge_triggered: true) for false→true transitions only (module API).

Regra.estatisticas(rule_pid)
# => %{notificacoes: ..., execucoes: ...}

Use Process.sleep/1 sparingly in tests; for deterministic waits, the optional Service below can help drain work.

Optional: `Service` for registration and global stats

Tec0301Pon.PON.Service is not started by Application. If you start it yourself, you can register fact names and rule pids, then query aggregated counters (estatisticas_globais/0) or call wait_until_queues_empty/1 in tests. It does not participate in the notification protocol itself—it is observability glue.

{:ok, _} = Tec0301Pon.PON.Service.start_link()
Tec0301Pon.PON.Service.registrar_fato(:counter, :counter)
Tec0301Pon.PON.Service.registrar_regra(:my_rule, rule_pid)
Tec0301Pon.PON.Service.estatisticas_globais()

What we defer to later parts

defrule / defpremissa and generated modules — Part 3 on dev.to (metaprogramming and less boilerplate).
Ports & adapters — Part 4 on dev.to (keeping side effects behind boundaries).
Message storms, coalescing, ETS tuning — Part 10 on dev.to; Part 11 on dev.to (profiling).

Summary

Facts are named GenServers; the Registry Tec0301Pon.PON.PubSub is a duplicate-key topic bus. On change, a fact dispatches to all registered pids with {:notificacao, name, value}. Rules and premises are subscriber processes that register on fact keys and update local state or downstream facts. That is the whole whiteboard-to-OTP bridge in this PoC.

References and further reading

Elixir Registry — duplicate keys, dispatch patterns — HexDocs.
Elixir GenServer — cast/call, state — HexDocs.
Armstrong (2003) — Making reliable distributed systems… — thesis PDF.
Cesarini & Thompson — Programming Erlang (2nd ed.) — OTP supervision and process design.
In this repo — Bibliography on dev.to — PON + Smart Brewery series (EN drafts) · repo draft; Code map: lib/tec0301_pon/application.ex, lib/tec0301_pon/pon/fato.ex, lib/tec0301_pon/pon/regra.ex, lib/tec0301_pon/pon/premissa.ex, lib/tec0301_pon/pon/service.ex, lib/tec0301_pon/pon/fanout.ex.

Published on dev.to: From whiteboard to code: mapping Facts, Rules, and Premises to OTP processes — tracked in docs/devto_serie_pon_smart_brewery.md.

Previous: Part 1 on dev.to — PON in Elixir: why the BEAM fits reactive rules · repo draft

Next: Part 3 on dev.to — A metaprogrammed DSL: defrule and defpremissa with less PON boilerplate · repo draft. Author hub: dev.to/matheuscamarques; URLs tracked in docs/devto_serie_pon_smart_brewery.md.

DEV Community: Matheus de Camargo Marques

Bibliography — PON + Smart Brewery dev.to series (EN drafts)

Bibliography — PON + Smart Brewery dev.to series (EN drafts)

Paradigms and architecture

Erlang, Elixir, OTP

Phoenix ecosystem

Data pipelines and observability

Analytics and BI

Digital twins and industrial context

Machine learning lifecycle

Queues, performance, back-pressure

This repository (deep dives in Portuguese or internal)

Retrospective: lessons from building a reactive rules engine in Elixir

Retrospective: lessons from building a reactive rules engine in Elixir

What the twelve parts built

What worked well

Trade-offs we accepted

What I would try earlier next time

Code spine (the same system, twelve lenses)

End-to-end picture

Series index (parts 1–12 + consolidated bibliography)

References and further reading (series-level)

Thank you

Dev profiling: CPU, memory, and what changed after optimizations

Dev profiling: CPU, memory, and what changed after optimizations

What profilers actually show

Reproducible load: PipelineWorkload

Two modes matter for interpretation

Environment variables (cheat sheet)

Quick commands

60-second battery

Profiling the Monte Carlo GenServer itself

Baselines: “light” vs “heavy” server

Optimizations this repo ties to measurements

BI / TSDB sanity (one paragraph)

Flow under profiling

Summary

References and further reading

When notifications explode: message storms, deduplication, and back-pressure in PON

When notifications explode: message storms, deduplication, and back-pressure in PON

Symptom: CPU spent orchestrating, not deciding

1. Deduplicate at the source (Fato)

2. Coalesce across facts (Fanout + atualizar_lote/1)

3. Drain the rule mailbox before evaluating (drain_notificacoes/3)

4. Fast reads without blocking the fact (ETS)

5. Back-pressure outside the engine (Phoenix / Broadway)

Message contract cheat sheet

Measuring (preview — full walkthrough in Part 11 on dev.to)

Flow diagram

Summary

References and further reading

ML on the digital twin: export, train pilots, and import predictions back into the app

ML on the digital twin: export, train pilots, and import predictions back into the app

Closed loop in three hops

Step 1: mix export.ml

Step 2a: Elixir pilots (Scholar / Axon)

Step 2b: JSONL format for import

Step 3: persist predictions — schema and context

Mix task: import from disk

LiveView: /smart-brewery/ml-predictions

Flow diagram

Summary

References and further reading

BI without mystery: dimensions, facts, and consuming the data (e.g. Power BI)

BI without mystery: dimensions, facts, and consuming the data (e.g. Power BI)

Why not query only telemetry_events?

Dimension tables: equipment and variables

Calendar dimension and fact views

Event facts beside telemetry

Native BI in Elixir: SmartBreweryBI

Governance: read-only database role

Optional: push rows to Power BI REST

Verify the model: mix verify.bi

Architecture sketch

Summary

References and further reading

From simulation to storage: telemetry, Broadway/GenStage, and TimescaleDB

From simulation to storage: telemetry, Broadway/GenStage, and TimescaleDB

Why a separate persistence path?

Turning the warehouse on: supervision and environment

Reproducible load: `PipelineWorkload`

1. Deduplicate at the source (`Fato`)

2. Coalesce across facts (`Fanout` + `atualizar_lote/1`)

3. Drain the rule mailbox before evaluating (`drain_notificacoes/3`)

4. Fast reads without blocking the fact (`ETS`)

Step 1: `mix export.ml`

LiveView: `/smart-brewery/ml-predictions`

Why not query only `telemetry_events`?

Native BI in Elixir: `SmartBreweryBI`

Verify the model: `mix verify.bi`

`mount`: snapshot + subscriptions + one stream

From PON notifications to PubSub: `SmartBreweryFactBroadcaster`

First coalescing layer: `LiveViewEventBatcher`

Second coalescing layer: LiveView `pending_fato_updates`

Scripted walkthrough: `simular/0`

Ports in Elixir: `@callback` modules

Adapters: `@behaviour` + `@impl true`

Rules that call adapters: `PredioInteligente`

A metaprogrammed DSL: `defrule` and `defpremissa` with less PON boilerplate

What `defrule` expands to (conceptually)

String `when:` (discouraged for most apps)

What `defpremissa` expands to

Bonus: `defcondicao` — aggregate premises