<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Arthur Jin</title>
    <description>The latest articles on DEV Community by Arthur Jin (@arthur_jin_2aae6a72b94c25).</description>
    <link>https://dev.to/arthur_jin_2aae6a72b94c25</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4241897%2Ff6a66b0b-9e78-4db5-b7d7-9e3249040343.png</url>
      <title>DEV Community: Arthur Jin</title>
      <link>https://dev.to/arthur_jin_2aae6a72b94c25</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/arthur_jin_2aae6a72b94c25"/>
    <language>en</language>
    <item>
      <title>What Presidio misses for Canadian PII, and what I did about it</title>
      <dc:creator>Arthur Jin</dc:creator>
      <pubDate>Thu, 12 Nov 2026 23:03:48 +0000</pubDate>
      <link>https://dev.to/arthur_jin_2aae6a72b94c25/what-presidio-misses-for-canadian-pii-and-what-i-did-about-it-2c2</link>
      <guid>https://dev.to/arthur_jin_2aae6a72b94c25/what-presidio-misses-for-canadian-pii-and-what-i-did-about-it-2c2</guid>
      <description>&lt;p&gt;I'm Arthur, a systems engineer at Geek+ in Toronto, where I work on warehouse automation integrations. This post is about &lt;a href="https://github.com/ArthurJZL/canpii" rel="noopener noreferrer"&gt;canpii&lt;/a&gt;, an open-source package I wrote that adds Canadian identifiers to &lt;a href="https://github.com/data-privacy-stack/presidio" rel="noopener noreferrer"&gt;Microsoft Presidio&lt;/a&gt;, a widely used open-source PII detector.&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem, in one support ticket
&lt;/h2&gt;

&lt;p&gt;Here is a made-up ticket. Every identifier in this post is synthetic.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Ticket from Jane Tremblay: the clinic rejected my health card, OHIP 1234-567-897 AB.
Please also update our business number to 123456782 RT 0001.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And one in French:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Bonjour, mon numéro d'assurance maladie est BOUF 9451 1419 et la pharmacie le refuse.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The first contains an Ontario health number (OHIP) and a CRA Business Number with a GST/HST program account. The second contains a Québec health insurance number (RAMQ).&lt;/p&gt;

&lt;p&gt;I ran both through stock Presidio 2.2.366: the default recognizers, nothing added, with the small spaCy models for English and French, and a score threshold of 0.4. This is what it found in the English ticket:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;PERSON        'Jane Tremblay'   0.85
PHONE_NUMBER  '1234-567-897'    0.40
DATE_TIME     '123456782'       0.85
US_PASSPORT   '123456782'       0.40
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;After anonymization:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Ticket from &amp;lt;PERSON&amp;gt;: the clinic rejected my health card, OHIP &amp;lt;PHONE_NUMBER&amp;gt; AB.
Please also update our business number to &amp;lt;DATE_TIME&amp;gt; RT 0001.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The health number is masked, but as a phone number, and its version code &lt;code&gt;AB&lt;/code&gt; is left behind. The Business Number is masked as a date, and the program account &lt;code&gt;RT 0001&lt;/code&gt; is left behind. If your policy says "block health numbers" or "allow phone numbers", it does the wrong thing in both cases.&lt;/p&gt;

&lt;p&gt;In the French ticket, stock Presidio found nothing. The RAMQ number went through unchanged.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Canadian identifiers are awkward
&lt;/h2&gt;

&lt;p&gt;There is no single "Canadian health number". Each province issues its own:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Ontario: 10 digits plus an optional one- or two-letter version code.&lt;/li&gt;
&lt;li&gt;Québec: 4 letters and 8 digits, with the holder's birth date embedded.&lt;/li&gt;
&lt;li&gt;British Columbia: 10 digits starting with 9, with a MOD-11 check digit.&lt;/li&gt;
&lt;li&gt;Alberta: 9 digits, with no published check digit.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So one entity needs several recognizers, and they differ in how much you can trust a match. A BC number that passes its checksum is strong evidence. A 9-digit Alberta number is just nine digits; only the words around it ("AHCIP", "health number") tell you what it is. Passport numbers (2 letters and 6 digits) and bank account numbers (7 to 12 digits) are in the same position: without context they look like order IDs.&lt;/p&gt;

&lt;p&gt;Then there is French. The context words that raise a score have to exist in both languages: "carte santé", "assurance maladie", "permis de conduire", "numéro d'entreprise".&lt;/p&gt;

&lt;p&gt;Context matching has its own trap. Presidio does ship a Canadian SIN recognizer (disabled by default), and one of its context words is "sin". Presidio's default matching is by substring, and "sin" is a substring of "bu*&lt;em&gt;sin&lt;/em&gt;*ess". With that recognizer enabled:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Number on file: 130-692-544.             CA_SIN 0.5
Our business number is 130-692-544.      CA_SIN 0.85
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The word "business" raises the SIN score from 0.5 to 0.85, as much as the word "SIN" itself does. I reported this upstream as &lt;a href="https://github.com/data-privacy-stack/presidio/issues/2459" rel="noopener noreferrer"&gt;presidio#2459&lt;/a&gt;. canpii's analyzer uses whole-word context matching, where the second sentence scores 0.5.&lt;/p&gt;

&lt;h2&gt;
  
  
  What canpii does
&lt;/h2&gt;

&lt;p&gt;canpii adds five entities in Presidio's own style: &lt;code&gt;CA_HEALTH_NUMBER&lt;/code&gt; (Ontario, Québec, BC, Alberta), &lt;code&gt;CA_PASSPORT&lt;/code&gt;, &lt;code&gt;CA_BN&lt;/code&gt;, &lt;code&gt;CA_BANK_ACCOUNT&lt;/code&gt; and &lt;code&gt;CA_DRIVER_LICENSE&lt;/code&gt; (Ontario, Québec, BC). Each recognizer has English and French context words, and validation where an official rule exists: the BC checksum, the RAMQ birth date, the CRA program identifiers. It also turns on Presidio's own SIN and postal code recognizers.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;presidio_anonymizer&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;AnonymizerEngine&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;canpii&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;create_analyzer&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;get_preset&lt;/span&gt;

&lt;span class="n"&gt;analyzer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;create_analyzer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;languages&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"en"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"fr"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;entities&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;get_preset&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"pipeda"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;entity_list&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;tickets&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="s2"&gt;"en"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"Ticket from Jane Tremblay: the clinic rejected my health card, "&lt;/span&gt;
        &lt;span class="s2"&gt;"OHIP 1234-567-897 AB. Please also update our business number to 123456782 RT 0001."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"fr"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Bonjour, mon numéro d'assurance maladie est BOUF 9451 1419 et la pharmacie le refuse."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;language&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;tickets&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;results&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;analyzer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;analyze&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;language&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;language&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;entities&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;entities&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;AnonymizerEngine&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;anonymize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;analyzer_results&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Output:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Ticket from &amp;lt;PERSON&amp;gt;: the clinic rejected my health card, OHIP &amp;lt;CA_HEALTH_NUMBER&amp;gt;. Please also update our business number to &amp;lt;CA_BN&amp;gt;.
Bonjour, mon numéro d'assurance maladie est &amp;lt;CA_HEALTH_NUMBER&amp;gt; et la pharmacie le refuse.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The version code and the program account are now part of the match, and the French ticket is handled. &lt;code&gt;pipeda&lt;/code&gt; is one of three presets (with &lt;code&gt;phipa&lt;/code&gt; and &lt;code&gt;quebec_law25&lt;/code&gt;): lists of entity types that are a starting point for a policy. They are not legal advice.&lt;/p&gt;

&lt;h2&gt;
  
  
  Numbers
&lt;/h2&gt;

&lt;p&gt;The repository has a benchmark: 1,740 bilingual template sentences with 700 labeled occurrences of the five identifiers above, some with a context word and some without. I ran the positives through stock Presidio and through Presidio + canpii at threshold 0.4. Recall:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Entity&lt;/th&gt;
&lt;th&gt;Gold spans&lt;/th&gt;
&lt;th&gt;Stock: right entity&lt;/th&gt;
&lt;th&gt;Stock: flagged as anything&lt;/th&gt;
&lt;th&gt;canpii: right entity&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;CA_HEALTH_NUMBER&lt;/td&gt;
&lt;td&gt;240&lt;/td&gt;
&lt;td&gt;0.000&lt;/td&gt;
&lt;td&gt;0.696&lt;/td&gt;
&lt;td&gt;0.863&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;CA_PASSPORT&lt;/td&gt;
&lt;td&gt;60&lt;/td&gt;
&lt;td&gt;0.000&lt;/td&gt;
&lt;td&gt;0.183&lt;/td&gt;
&lt;td&gt;0.667&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;CA_BN&lt;/td&gt;
&lt;td&gt;60&lt;/td&gt;
&lt;td&gt;0.000&lt;/td&gt;
&lt;td&gt;0.500&lt;/td&gt;
&lt;td&gt;0.933&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;CA_BANK_ACCOUNT&lt;/td&gt;
&lt;td&gt;160&lt;/td&gt;
&lt;td&gt;0.000&lt;/td&gt;
&lt;td&gt;0.506&lt;/td&gt;
&lt;td&gt;0.625&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;CA_DRIVER_LICENSE&lt;/td&gt;
&lt;td&gt;180&lt;/td&gt;
&lt;td&gt;0.000&lt;/td&gt;
&lt;td&gt;0.483&lt;/td&gt;
&lt;td&gt;0.778&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;All five&lt;/td&gt;
&lt;td&gt;700&lt;/td&gt;
&lt;td&gt;0.000&lt;/td&gt;
&lt;td&gt;0.537&lt;/td&gt;
&lt;td&gt;0.776&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;"Flagged as anything" is the generous reading for stock Presidio: any part of the identifier tagged as any entity, as in the ticket above. It covers the whole identifier for only 0.360 of the spans.&lt;/p&gt;

&lt;p&gt;The caveat matters more than the table. I wrote the template sentences, so this is a regression benchmark, not an estimate of accuracy on real text. Recall is also only half of it: on the same corpus canpii's precision ranges from 0.71 (bank numbers) to 1.00. The method and full tables are in &lt;a href="https://github.com/ArthurJZL/canpii/blob/main/benchmarks/RESULTS.md" rel="noopener noreferrer"&gt;benchmarks/RESULTS.md&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where it falls short
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;No context, no match, for the weak patterns.&lt;/strong&gt; All of canpii's misses in that table are sentences without a context word. There it found 0 of 20 passport numbers and 0 of 60 bank numbers. That is deliberate, because flagging every 7-digit number is worse, but it means free-form text will have misses.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Provinces are missing.&lt;/strong&gt; Health cards for Saskatchewan, Manitoba and Nova Scotia, and driver's licences for Alberta and Manitoba, are not covered yet. Neither are New Brunswick, PEI, Newfoundland and Labrador or the territories.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The French needs a native review.&lt;/strong&gt; The French context words and test sentences have not been reviewed by someone who writes Québec French every day.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The benchmark is synthetic.&lt;/strong&gt; A more realistic corpus is an open issue.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Issues &lt;a href="https://github.com/ArthurJZL/canpii/issues" rel="noopener noreferrer"&gt;#1 to #8&lt;/a&gt; cover these gaps. Each province is a self-contained recognizer, validator and tests, with a recipe in CONTRIBUTING.md. If you know your province's card format, that is a small pull request. Two rules: every format rule cites a source, and no real identifiers, ever.&lt;/p&gt;

&lt;h2&gt;
  
  
  Links
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Live demo (synthetic examples): &lt;a href="https://canpii.streamlit.app" rel="noopener noreferrer"&gt;https://canpii.streamlit.app&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Repository: &lt;a href="https://github.com/ArthurJZL/canpii" rel="noopener noreferrer"&gt;https://github.com/ArthurJZL/canpii&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;PyPI: &lt;a href="https://pypi.org/project/canpii/" rel="noopener noreferrer"&gt;https://pypi.org/project/canpii/&lt;/a&gt; (&lt;code&gt;pip install canpii&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Upstream context bug: &lt;a href="https://github.com/data-privacy-stack/presidio/issues/2459" rel="noopener noreferrer"&gt;https://github.com/data-privacy-stack/presidio/issues/2459&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>python</category>
      <category>privacy</category>
      <category>opensource</category>
      <category>llm</category>
    </item>
  </channel>
</rss>
