I follow the small municipalities around my region in Brazil. Everything they buy is published as a text description in a national procurement API, and a share of those purchases is IT work I can bid on: maintenance contracts, network cabling, software licenses, toner, a hosted portal. The API has no full text search worth using, so I wrote a filter over the object description. Last run made 424 requests across four procurement modalities, found 165 objects containing an IT term, and marked 12 of them as billable under the MEI occupations I hold today.
The filter is a keyword classifier. It has two ways to be wrong, and they cost very different things. A false positive costs me a minute of reading. A false negative costs the opportunity and leaves no trace in the run. Everything below came from a real object that fooled it.
Accents first, because the source is Portuguese
Object descriptions arrive in uppercase, with and without accents, and often carry a platform prefix like [Portal de Compras Publicas] -. Both get normalized before any matching:
def sem_acento(txt):
n = unicodedata.normalize('NFKD', txt or '')
return ''.join(c for c in n if not unicodedata.combining(c)).lower()
def limpa_objeto(txt):
"""Remove prefixos de plataforma tipo '[Portal de Compras Publicas] - '."""
t = re.sub(r'^\s*\[[^\]]{0,60}\]\s*-?\s*', '', (txt or '').strip())
return re.sub(r'\s+', ' ', t).strip()
NFKD decomposition plus a filter on combining characters turns Manutenção into manutencao. Without it, half my keyword list never matches, and the failure is invisible because a filter that finds nothing looks like a quiet week.
Short terms need a word boundary
Seventy-four terms make up the strong list. Most are long enough that a plain substring test is correct: computador should match computadores, and that inflection is the normal case in Portuguese. Short terms behave differently, because they live inside other words.
for termo in FORTES:
if len(termo) <= 3:
if re.search(r'\b' + re.escape(termo) + r'\b', t):
achados.append(termo)
elif termo in t:
achados.append(termo)
One term in the list is short enough to need it: ti. A plain substring test on those two letters fires on licitacao, noticia and importante, which appear in most of a procurement feed. Everything longer stays on the substring test, rack included, because computador has to match computadores and that inflection is the rule rather than the exception.
Ambiguous words need a companion in the same sentence
Four terms are real IT vocabulary and also everyday Portuguese: servidor (server, or civil servant), suporte tecnico (IT support, or support for anything), cartucho (printer cartridge, or respirator cartridge), licenca (software license, or a professional registration fee). Those four never classify an object on their own:
if all(a in TERMOS_FRACOS for a in achados):
if not any(c in t for c in COMPANHIA_IT):
return achados, 'termo ambiguo sem companhia de TI'
The companion list has 53 entries, from impressora to cabeamento. The rule was added after a measurement on 19/10/2026 over 173 findings: without it, 15 objects entered by mistake, among them medical expert examinations, uniforms, a marmitex lunch contract, a water cooler, an LED panel rental, and two consultancy contracts in social assistance and public transparency. No legitimate object was lost by adding it.
That measurement is the reason I trust the rule. Fifteen wrong entries in 173 was enough to act on, and losing no legitimate object is what made the rule safe to keep.
The reject list is a log with dates
Eighty-three entries exist to kill a match even when a strong term is present. Each one came from a specific sentence:
-
tomografia computadorizadamatchedcomputador. -
filmes radiologicos para impressora de imagens medicasmatchedimpressora. -
leite integral UHT para pacientes, acompanhantes e servidores da Fundacaomatchedservidor. -
independente de transcricaois boilerplate at the end of a metalwork contract and matchedtranscricao. -
licenca anual de Anotacao de Responsabilidade Tecnicamatchedlicenca. -
linha fixafor a telephone line matched telephony, which is not an occupation of mine.
The comments in the file carry the date of the scan that produced each group. That turns the reject list into a record of what the source actually contains, and the next person editing it can see why a line exists instead of deleting it as noise.
One reject entry nearly killed a real opportunity
Civil servants are everywhere in this feed. A notice for a basic computing course aimed at municipal employees is a legitimate purchase I can invoice, and it contains the word servidor in its most common sense. The reject rule therefore has an exception:
for r in RUIDO:
if r in t:
if r in RUIDO_SERVIDOR and any(f in t for f in SERVICO_DE_MEI):
continue
return achados, r
If the noise belongs to the group about people, and the same object also matches a service I actually sell, the object survives. Without those two lists, curso de informatica para os servidores municipais would be dropped by my own filter, silently, and it is one of the few things I can invoice.
The tests are a list of inputs that must be rejected
The test file holds 15 positive cases and 24 negative ones, and it fails the run when any of them misses. A negative case passes when the classifier finds no term at all, or when it finds terms and returns a reason to discard:
for nome, txt in negativo:
a, m = v.classifica(txt)
ok = (not a) or bool(m)
falhas += 0 if ok else 1
The negative list is the valuable half, because I wrote it by pasting objects that had already fooled the filter. AQUISICAO DE FILMES RADIOLOGICOS DRY FILM PARA IMPRESSORA DE IMAGENS MEDICAS is in there. So is a car rental that mentions computer maintenance, and a school lunch contract that mentions IT equipment.
One case checks the reason, not the outcome. A bank accreditation notice matches digitacao and would be labeled with my data entry occupation. The correct behavior is a discard by a specific noise entry, so the assertion names the entry:
achados, ruido = v.classifica(txt_banco)
ok = bool(achados) and ruido in ('instituicoes financeiras', 'arrecadacao de tributos',
'documento de arrecadacao municipal')
Written as "no profile, therefore fine", that test would pass for any future reason, including a rule that drops legitimate notices. The same notice, from Senador Modestino, had already been published as billable before the fix.
Recall fails without a message
On 16/11/2026 a Diamantina notice for aquisicao de computadores was marked as outside my MEI, which was wrong, since buying computers falls under a commerce occupation I can use. The fix added four phrases that name the purchase explicitly: aquisicao de computadores, fornecimento de computadores, aquisicao de microcomputadores, aquisicao de computadores e perifericos.
I did not add computador alone to that list, even though it would have caught the case. It would also match spare parts, maintenance and rental, and those are different occupations. The narrower fix costs a future miss and keeps the report honest.
That is the asymmetry I plan around. Precision errors show up in the output and get fixed in a minute. Recall errors do not show up at all, which is why the report is read every week instead of trusted. The test suite catches what I thought of. Reading the notices catches what I did not.
Today the filter has 74 strong terms, 53 companion terms and 83 noise entries. Run against the current snapshot of 164 objects, none of them would be dropped by the rules as they stand, and the control suite finishes with zero failures. None of that says the next municipality uses a phrase I have not seen.
Se você mantém algum filtro de palavra-chave em cima de texto de terceiro, comenta qual foi o primeiro falso positivo que te obrigou a escrever um teste. Eu respondo em português aqui embaixo.
Top comments (0)