DEV Community: Ivor

Fail-Safe PII Redaction in Elixir

Ivor — Sat, 26 Apr 2025 11:52:32 +0000

Filter by Value, Not by Path

In this post, I use the example of a parcel-shipping gateway integrating with multiple carriers.
This example is purely hypothetical and chosen to clearly illustrate the technical idea.

Integrating with N shipping carriers often means dealing with N different JSON (if you’re lucky!) responses.

Each carrier’s responses can sprinkle customer PII into your logs—until your CISO sees it.

Most teams try to predict where the PII will be:

"recipient.email" for Carrier A
"ship_to.contact.email" for Carrier B
"error.stacktrace[0].ctx.email" when Carrier C blows up 😱

That game never ends.

Let’s flip the problem: we already know the values before we send any request.

So let’s redact by value, not by path.

Step 1 — Our Canonical Parcel Struct

defmodule Parcel do
  @enforce_keys ~w(id name email phone)a
  defstruct [:id, :name, :email, :phone, :address, :items]
end

Everything downstream is derived from %Parcel{}.

Step 2 — Collect the Secrets Once

defmodule Secrets do
  @doc """
  Accepts a Parcel struct and returns a list of 
  values that are secrets that we don't want to see in our logs.
  """
  @spec from_parcel(Parcel.t()) :: [String.t()]
  def from_parcel(%Parcel{} = p) do
    [
      p.name,
      p.email,
      p.phone,
      p.address.street,
      p.address.city,
      p.address.postcode
    ]
    |> Enum.reject(&is_nil/1)
  end
end

Step 3 — A Drop-in Logger Helper

defmodule SafeLog do
  require Logger

  @doc """
  Logs a payload at the given level, safely redacting any known secret values.

  If the payload is a map, it is encoded as JSON before logging.
  If the payload is a printable string, it is used directly.
  """
  @spec scrub_and_log(
          :debug | :info | :warn | :error,
          String.t(),
          map() | String.t(),
          [String.t()]
        ) :: :ok
  def scrub_and_log(level, message, payload, secrets)

  def scrub_and_log(level, message, payload, secrets) when is_map(payload) do
    payload
    |> Jason.encode!()
    |> redact_and_log(level, message, secrets)
  end

  def scrub_and_log(level, message, payload, secrets) when is_binary(payload) do
    if String.printable?(payload) do
      redact_and_log(level, message, payload, secrets)
    else
      Logger.warn("SafeLog attempted to log a non-printable binary, skipping.")
      :ok
    end
  end

  @spec redact_and_log(
          :debug | :info | :warn | :error,
          String.t(),
          String.t(),
          [String.t()]
        ) :: :ok
  defp redact_and_log(level, message, text, secrets) do
    redacted =
      secrets
      |> Enum.uniq()
      |> Enum.sort_by(&String.length/1, :desc)
      |> Enum.reduce(text, fn secret, acc ->
        String.replace(acc, secret, "[FILTERED]")
     end)

    Logger.log(level, fn -> "#{message} #{redacted}" end)
  end
end

Step 4 — Use It in an Adapter

defmodule ShippingExpressAdapter do
  @doc """
  Creates a shipping label by sending a request to the ShippingExpress API.

  Logs the request and response with PII safely scrubbed out,
  using the known secret values extracted from the Parcel struct.
  """
  @spec create_label(Parcel.t()) :: {:ok, map()} | {:error, any()}
  def create_label(%Parcel{} = parcel) do
    secrets = Secrets.from_parcel(parcel)
    request = build_request(parcel)

    SafeLog.scrub_and_log(:debug, "ShippingExpress request ⇢", request, secrets)

    with {:ok, raw_resp} <- HTTPClient.post(url(), request),
         {:ok, resp}     <- Jason.decode(raw_resp) do
      SafeLog.scrub_and_log(:debug, "ShippingExpress response ⇠", resp, secrets)
      parse_label(resp)
    end
  end
end

No carrier-specific filter modules.

If ShippingExpress decides to throw "InternalError: phone=123-456" somewhere, it still gets nuked.

Considerations

Concern	Mitigation
Large payloads?	Replacing many secrets in large payloads could introduce small slowdowns. Benchmark if you process very large logs.
Secrets overlapping or partial matches?	Deduplicate secrets (`Enum.uniq/1`) and sort them by descending length (`Enum.sort_by/2`) before replacing. This ensures longer secrets are redacted first, avoiding partial leaks (e.g., preventing `"john.smith@example.com"` from becoming `"[FILTERED].smith@example.com"`).
Encoding issues	If sensitive values are encoded (e.g., Base64) before being logged, simple string replacement won't find them. Make sure sensitive fields are logged in plain form, or exclude them entirely.

If you can think of scenarios where this approach might fail, I'd love to learn from your experience.

Conclusion

Third-party APIs are unpredictable — but the data you send is something you control.

Building your filters around known values, rather than brittle paths, can make your systems more robust and easier to trust.

OpenAI's Agent building guide - summary

Ivor — Sun, 20 Apr 2025 10:46:35 +0000

On Thursday, 17 April 2025, OpenAI published their PDF on building agents: "A Practical Guide to Building Agents"

I took some time on the weekend to read through it and see if I can distill some information from here.

Here are my take-aways.

Workflow: Steps taken towards a goal. Some steps are deterministic and some are non-deterministic.

Traditional Software: Digitised, streamlined and deterministic steps that are stitched together by using humans for the non-deterministic steps to complete a workflow.

Agents: Independent, non-deterministic steps stitched together by tool calls, which may or may not be deterministic (traditional software or other agents).

Examples of non-agent software using machine learning:

Translation

Transcription

Labelling

Sentiment analysis

Summary generation

These are all single steps that humans can execute. Humans would work towards a goal, use these pieces of software to gather information and manipulate the world in order to reach some end goal.

Agents can take over some of the non-deterministic interpretation of steps that are currently made by humans.

Agents can, in non-deterministic settings:

make decisions
recognise when a goal is reached
recognise when they need to make a course correction
halt execution and involve a human

Tools: Access to the world beyond the agent. Either to gather context, or to take actions.

Examples:

internet APIs (weather api)

local APIs (computer use)

physical APIs (robotic limbs, alarm systems)

other agents

Potential use-cases for agents:
Workflows that previously resisted automation.

complex decisions
extensive rulesets with costly updates that can be error prone
unstructured data - hard for traditional software to interpret.

Agentic Cycle

This cycle represents the "run loop" that an agent cycles through, evaluating what it knows, determining if it needs more context or needs to take action and comparing its progress with its goal.

Agent components

Model: LLM driving the independent workflow.
Tools: Capabilities available to the agent.
Instructions: Guidelines and guardrails.

Model selection

Start smart, optimise when outcomes can be compared with smartest benchmark.
Considerations:

local vs hosted
context window
latency
cost

Tools

Documenting tools thoroughly make them discoverable to agents. Helps the model understand what they can be used for and what they should not be used for.

Three types:

Data gathering tools
Tools to take action in the world
Orchestration - tools (agents) to orchestrate other agents.

Instructions

Clarity:
Make instructions as clear as possible:

Reduce ambiguity
improve decisions
smoother workflow execution
fewer errors.

Strategy:
Instruct agent to break down tasks.

Be explicit about:

which actions to take
which tools to use.

Tool-use can ensure that deterministic steps are deterministic and avoid hallucinations.

Context and edge cases:
Preempt and explain known edge-cases.
Provide clean instructions for handling them.
Leverage existing business documents like customer support scripts, how-to guides and other instruction manuals.

Orchestration

Agents with a very complex workflow or that have access to a large number of tools are more prone to get confused.
Simpler agents with more focussed tools are better:

separation of concerns: less "cognitive" load
Simpler agents are easier to validate and optimise.
More composable agents allow for more dynamic workflows

Orchestration is allowing different agents to work together by making them known to each other and providing good documentation about what each agent is good at.

There are two types of orchestration:

Manager: One agent with access to the other agents as tools. Calls and waits on them as part of its own workflow.

Example: "Translate this text into French, Italian and Spanish". Manager will send the request to an agent focussed on each language. Collects the responses and returns the result to the user.

Decentralised agents: Agents that know about other agents as tools and can determine that the workflow should be handed over to a specific agent, terminating its involvement in the workflow.

Example:
Welcome agent can receive customer call, determine the nature of the call and transfer the request to an agent that can take over - agent to handle complaints, agent to handle refunds, etc.

Guardrails

Guardrails can be implemented as agents that evaluate the output of the agents working on reaching the goal.

Layered approach:
Start with general guards such as:

data privacy risks
reputation risks

Other types of guards:

relevance
safety
PII
moderation
tool risks
output validation

Lastly, guardrails can be added against specific real world cases where previous guards failed.

Optimistic execution: Guards can run concurrently with some of the steps and only intervene when their conditions are breached. At this point it might be necessary for human intervention.

Conclusion

Agents represent a new era where software can reason through ambiguity and navigate complex workflows autonomously.

Foundations of good models:

capable models
well-defined tools
clear instructions

Start simple, with a single agent.
Validate use-case.
Evolve into multi-agent orchestration when it becomes necessary.

How to unsubscribe from all topics in Phoenix.PubSub

Ivor — Fri, 18 Nov 2022 08:50:22 +0000

In this post I want to share a quick tip that I found very useful when relying on Phoenix.PubSub to reflect server side changes on the UI.

Phoenix.LiveView makes it wonderfully easy to build rich UIs without needing to use a big client-side javascript framework. The server does live tracking of variables, determines the diff based on those changes and pushes those to the client where the changes are applied.

This means that it is easy to build a page where live updates can be monitored without needing to reload a page. An example could be a dashboard to monitor the state of various servers on a network. They can all update the app via an API. Each time an update comes in we can broadcast the update to a topic for that server so that all the users viewing those servers will get the update, for example:

MyApp.PubSub
|> Phoenix.PubSub.broadcast("server-#{server_id}", %{cpu: update.cpu, mem: update.mem})

If we have a liveview that shows the state of the server with various stats about it, we can subscribe to the topic to update the stats as they come in.

Phoenix.PubSub.subscribe(MyApp.PubSub, "server-#{server.id}")

We can do this in the mount/3 callback of the liveview, but mount is not called when you patch so instead we can do the subscribe in the handle_params/3 callback which is called every time the browser location is changed.

To make for a snappy experience, we can add a dropdown on the page to switch to a server with a different id. By using patch instead of redirect we can swop out the server we're monitoring without having to load a new page and without the need for a new liveview process.

The problem is that if you subscribe to the new server in the handle_params/3 callback you are now subscribed to two servers and will get updates for each of them.
It would be great if you can unsubscribe from the previous server. In this simple case you can probably unsubscribe from the previous one by referring to the server variable that is currently in the socket assigns. But, you may not always have that luxury. And in some cases you might have also subscribed a number of other related topics, and some of them maybe conditionally depending on the attributes of the server. Maybe a database server will get server updates but also database specific updates etc.

A function that allows you to see all the topics that the current process is subscribed to will be very handy at this point. I could not find such a function in Phoenix.PubSub, but, looking at the source for the subscribe/3 and unsubscribe/2functions we can see how to achieve this result.

Phoenix.PubSub uses the Registry module under the hood. The name of the PubSub instance that you started in your application's Application.start/2 callback is the registry.

  ...
  {Phoenix.PubSub, name: MyApp.PubSub}, # this is the registry
  ...

The key for the registry entry is the topic that is being subscribed to.

Phoenix.PubSub.subscribe(pubsub, topic, opts \\ [])

is (pretty much)

Registry.register(registry, key, value)

If the key is the topic we can find all the topics that our process is subscribed to by calling

Registry.keys(MyApp.PubSub, self())

which returns a list of the topics.

In order to start with a fresh set of subscriptions and avoid double subscribing or mistaking updates for one server for updates to a previous one, we can unsubscribe from all these topics before subscribing to the new one:

  defp unsubscribe_all() do
    MyApp.PubSub
    |> Registry.keys(self())
    |> Enum.map(fn topic ->
      Phoenix.PubSub.unsubscribe(MyApp.PubSub, topic)
    end)
  end

  def handle_params(%{"id" => server_id}, _url, socket) do
    unsubscribe_all()
    MyApp.PubSub
    |> Phoenix.PubSub.subscribe("server-#{server_id}")

    socket = assign(socket, :server, fetch_server(server_id))

    {:noreply, socket}
  end

The ability to know which subscriptions a process has seems like a useful function. I am curious to know if this is useful to others or if there are problems with dropping down to the registry and retrieving this information.

Beware Ecto's "or_where" pitfall

Ivor — Wed, 29 Jun 2022 15:12:07 +0000

In this post I want to highlight an error that can creep into your use of Ecto's or_where function. This is not a fault with Ecto, it is merely a way in which we can jump to the wrong conclusion about how this function works in conjunction with Ecto.Query functions.

Every now and then I find myself building a form to filter a specific resource. This form allows users to filter records where a specific attribute is equal to a certain value (think select), or where a specific attribute is in a list of values (think multi-select).

As an example, return all the medicines that are tablets, and treat either nausea or a headache. This means that records that are not tablets should not be returned. And tablets that do not have "nausea" or "headache" in the list of conditions that they can treat, should not be returned. In order to add an AND clause to all the queries, to highlight how this breaks, I also add a boolean field available. We will include a clause to filter out any unavailable records to all our queries.

Here is the relevant part of the schema:

schema "medicines" do
  ...
  field :type, :string
  field :conditions, {:array, :string}
  field :available, :boolean, default: false
end

The first part is easy. We return records where type is tablet:

defmodule MedicineCupboard.MedicineFilter do
  import Ecto.Query

  def type_filter(query, %{"type" => type}) do
    where(query, [m], m.type == ^type)
  end
  def type_filter(query, _no_type), do: query
end

Starting with the available clause we get this SQL:

SELECT m0."id", m0."name", m0."type", m0."conditions", m0."available" 
FROM "medicines" AS m0 WHERE 
(m0."available" = TRUE) AND (m0."type" = $1) ["tablet"]

The second filter is a bit more complicated. We want to return medicines where either of the the values passed as the condition filter is present in the conditions array of strings.

It is tempting to write this second filter the way it is written below:

defmodule MedicineCupboard.MedicineFilter do
  ...

  # This does not build the intended query!
  def condition_filter(query, %{"condition" => conditions_list}) do
    conditions_list
    |> Enum.reduce(query, fn condition, query -> 
      or_where(query, [m], ^condition in m.conditions)
    end)
  end
end

The problem with this query is that the SQL that it builds will look like this:

SELECT m0."id", m0."name", m0."type", m0."conditions", m0."available" 
FROM "medicines" AS m0 WHERE 
((m0."available" = TRUE)) 
OR ($1 = ANY(m0."conditions")) 
OR ($2 = ANY(m0."conditions")) ["nausea", "headache"]

The Ecto query looks like this:

#Ecto.Query<from m0 in MedicineCupboard.Medicine, where: m0.available == true,
 or_where: ^"nausea" in m0.conditions, or_where: ^"headache" in m0.conditions>

Any medicine that includes headache in the conditions field, whether it is available or not will be returned. The clause for availability is ORed with the clauses for the conditions.

What we want to have is available = true AND nausea in conditions, or headache in conditions.

The Ecto.Query or_where/3 function appends an OR clause to all the previous conditions, effectively resulting in:
Either satisfy all the conditions up to this point, OR just satisfy this new or_where clause.

A way that I have solved this is to use a subquery:

defmodule MedicineCupboard.MedicineFilter do
  import Ecto.Query

  def type_filter(query, %{"type" => type}) do
    where(query, [m], m.type == ^type)
  end
  def type_filter(query, _no_type), do: query

  def condition_filter(query, %{"condition" => condition_list}) do
    conditions_query = condition_list
    |> Enum.reduce(Medicine, fn condition, query -> 
      or_where(query, [m], ^condition in m.conditions)
    end)
    |> select([:id])

    where(query, [m], m.id in subquery(conditions_query))
  end
end

This groups the various or_where clauses in one subquery and then appends an AND where clause to the incoming query, where we return only records where their id is returned by the conditions subquery.

The resulting SQL looks like this:

SELECT m0."id", m0."name", m0."type", m0."conditions", 
m0."available" FROM "medicines" AS m0 WHERE 
(m0."available" = TRUE) 
AND 
(m0."id" IN 
  (SELECT sm0."id" FROM "medicines" AS sm0 WHERE 
    ($1 = ANY(sm0."conditions")) OR ($2 = ANY(sm0."conditions"))
  )
) 
["nausea", "headache"]

and the Ecto query looks like this:

#Ecto.Query<from m0 in MedicineCupboard.Medicine, where: m0.available == true,
 where: m0.id in subquery(#Ecto.Query<from m0 in MedicineCupboard.Medicine, or_where: ^"nausea" in m0.conditions, or_where: ^"headache" in m0.conditions, select: [:id]>)>

Lastly, we can now also combine the type and conditions filters and see that the available and type filter are joined with AND and that the conditions subquery, with it's embedded OR clauses, is also added to the existing queries with an AND.

params = %{"type" => "tablet", "conditions" => ["nausea", "headache"]}
Medicine
|> where([m], m.available == true)
|> MedicineFilter.type_filter(params)
|> MedicineFilter.condition_filter(params)
|> Repo.all()

This then gives us the following ecto query:

#Ecto.Query<from m0 in MedicineCupboard.Medicine, where: m0.available == true,
 where: m0.type == ^"tablet",
 where: m0.id in subquery(#Ecto.Query<from m0 in MedicineCupboard.Medicine, or_where: ^"nausea" in m0.conditions, or_where: ^"headache" in m0.conditions, select: [:id]>)>

and the SQL:

SELECT m0."id", m0."name", m0."type", m0."conditions", 
m0."available" FROM "medicines" AS m0 
WHERE (m0."available" = TRUE) 
AND (m0."type" = $1) 
AND (m0."id" IN (
  SELECT sm0."id" FROM "medicines" AS sm0 WHERE 
    ($2 = ANY(sm0."conditions")) OR ($3 = ANY(sm0."conditions"))
  )
) ["tablet", "fever", "headache"]

Hope this helps someone and if you have any comments on improvements, please feel free to share.

If you want to play around with the code, here is the sample project on github: https://github.com/Ivor/medicine_cupboard