DEV Community: tutorial

Construindo um Índice Invertido em Elixir: do zero ao TF-IDF

Matheus de Camargo Marques — Fri, 02 Oct 2026 14:35:30 +0000

Você já parou para pensar em como o Google, o Elasticsearch ou o Lucene conseguem encontrar documentos relevantes em milissegundos, mesmo com bilhões de páginas indexadas?

A resposta está em uma estrutura de dados elegante chamada índice invertido. E a boa notícia é que você pode implementá-la do zero em Elixir em menos de 100 linhas.

Neste artigo, vamos construir passo a passo um índice invertido, adicionar busca booleana (AND/OR), calcular frequência de termos (TF) e, por fim, ranquear resultados com TF-IDF.

O que é um índice invertido?

Se você pensar em um livro, o índice remissivo no final lista cada termo importante e as páginas onde ele aparece. Um índice invertido faz exatamente isso, mas para documentos:

"o gato subiu no telhado"   → doc 1
"o cachorro latiu"          → doc 2

Índice invertido:
%{
  "gato"     => [1],
  "subiu"    => [1],
  "telhado"  => [1],
  "cachorro" => [2],
  "latiu"    => [2]
}

Em vez de, para cada busca, varrer todos os documentos (custo O(n)), consultamos diretamente o mapa de termos — custo praticamente O(1).

Simples, né? Vamos codar.

Preparando o projeto

mix new indice_invertido
cd indice_invertido

Nada de dependências externas — só Elixir puro.

Passo 1: Tokenização

Antes de indexar, precisamos transformar texto bruto em tokens limpos: minúsculas, sem pontuação, sem stopwords. Esse é o trabalho da função tokenizar/1, e o módulo abaixo documenta cada decisão.

defmodule IndiceInvertido do
  @moduledoc """
  Módulo responsável por transformar texto bruto em tokens normalizados,
  etapa fundamental para a construção de um índice invertido.

  A função principal é `tokenizar/1`, que recebe uma string e devolve
  uma lista de tokens limpos — sem pontuação, sem maiúsculas e sem
  stopwords.

  ## O que são stopwords

  Stopwords são palavras tão comuns em um idioma que não ajudam a
  distinguir documentos entre si. Em português, palavras como "a",
  "o", "de", "para" aparecem em praticamente todo texto e, por isso,
  são descartadas durante a indexação. Mantê-las só aumentaria o
  tamanho do índice sem melhorar a qualidade da busca.

  As stopwords deste módulo são declaradas como atributo de módulo:

      @stopwords ~w(a o e de da do em no na um uma para com por que se)

  O sigil `~w(...)` (word list) é um atalho para criar uma lista de
  strings. Estas duas linhas são equivalentes:

      ~w(a o e de)
      ["a", "o", "e", "de"]

  ## Pipeline de tokenização

  A função `tokenizar/1` aplica quatro transformações em sequência,
  encadeadas com o operador pipe `|>`. O pipe pega o resultado da
  esquerda e passa como primeiro argumento da função da direita.
  Ou seja, `texto |> String.downcase()` é o mesmo que
  `String.downcase(texto)`.

  ### Etapa 1 — `String.downcase/1`

  Converte todo o texto para minúsculas, garantindo que "Gato",
  "gato" e "GATO" sejam tratados como o mesmo token.

      iex> String.downcase("O GATO Subiu")
      "o gato subiu"

  A função lida corretamente com caracteres acentuados:

      iex> String.downcase("AÇÃO")
      "ação"

  ### Etapa 2 — `String.replace/3` com o regex `~r/[^\\p{L}\\p{N}\\s]/u`

  Remove pontuação, símbolos e emojis, substituindo cada caractere
  indesejado por um espaço. O regex merece atenção:

    * `[^...]` — classe negada: "qualquer coisa que NÃO seja..."
    * `\\p{L}`  — qualquer letra Unicode (inclui `ç`, `é`, `日`)
    * `\\p{N}`  — qualquer número Unicode (inclui `0-9`)
    * `\\s`    — qualquer espaço em branco (espaço, tab, quebra de linha)
    * `/u`     — flag Unicode, indispensável para `\\p{...}` funcionar

  Em português: *casa com qualquer caractere que não seja letra,
  número ou espaço em branco*.

      iex> String.replace("olá, mundo!", ~r/[^\\p{L}\\p{N}\\s]/u, " ")
      "olá  mundo "

  A substituição é feita por **espaço**, e não por string vazia.
  Isso é importante: se apagássemos direto, `"fim.início"` viraria
  `"fimíncio"` (grudado). Com espaço, os termos permanecem separados.

      iex> String.replace("fim.início", ~r/[^\\p{L}\\p{N}\\s]/u, "")
      "fimíncio"

      iex> String.replace("fim.início", ~r/[^\\p{L}\\p{N}\\s]/u, " ")
      "fim início"

  Note que esta etapa pode gerar espaços duplos, o que é tratado
  na próxima.

  ### Etapa 3 — `String.split/3` com o regex `~r/\\s+/` e `trim: true`

  Divide a string em uma lista de tokens. O regex `\\s+` casa com
  um ou mais espaços em branco seguidos, então espaços duplos são
  tratados como um único separador. A opção `trim: true` descarta
  strings vazias no início e no final.

      iex> String.split("  olá  mundo  ", ~r/\\s+/)
      ["", "olá", "mundo", ""]

      iex> String.split("  olá  mundo  ", ~r/\\s+/, trim: true)
      ["olá", "mundo"]

  ### Etapa 4 — `Enum.reject/2` com o capture `&`

  Descarta os tokens que estão na lista de stopwords. `Enum.reject/2`
  percorre a lista e remove os elementos para os quais a função
  retorna `true` — é o oposto de `Enum.filter/2`.

      iex> Enum.reject([1, 2, 3, 4], fn x -> x > 2 end)
      [1, 2]

  O capture `&(&1 in @stopwords)` é uma forma abreviada de criar uma
  função anônima. `&1` refere-se ao primeiro argumento. Estas duas
  linhas são idênticas:

      Enum.reject(lista, &(&1 in @stopwords))
      Enum.reject(lista, fn token -> token in @stopwords end)

  O operador `in` testa se um item pertence a uma coleção:

      iex> "gato" in ["a", "o", "gato"]
      true

  ## Resultado final

      iex> IndiceInvertido.tokenizar("O gato subiu no telhado!")
      ["gato", "subiu", "telhado"]

      iex> IndiceInvertido.tokenizar("Ela comprou 3 maçãs, 2 peras e 1 banana!")
      ["ela", "comprou", "3", "maçãs", "2", "peras", "1", "banana"]

  Note que números são preservados por causa de `\\p{N}` no regex.
  """

  @stopwords ~w(a o e de da do em no na um uma para com por que se)

  def tokenizar(texto) do
    texto
    |> String.downcase()
    |> String.replace(~r/[^\p{L}\p{N}\s]/u, " ")
    |> String.split(~r/\s+/, trim: true)
    |> Enum.reject(&(&1 in @stopwords))
  end
end

Repare no uso de \p{L} e \p{N} com a flag u — isso garante que acentos como é e ã sejam preservados. Um detalhe importante ao trabalhar com português.

Teste rápido:

iex> IndiceInvertido.tokenizar("O gato subiu no telhado!")
["gato", "subiu", "telhado"]

Passo 2: Construindo o índice

Agora a parte central. Recebemos um mapa %{doc_id => texto} e retornamos %{termo => [doc_ids]}.

def construir(documentos) when is_map(documentos) do
  documentos
  |> Enum.flat_map(fn {doc_id, texto} ->
    texto
    |> tokenizar()
    |> Enum.uniq()
    |> Enum.map(fn termo -> {termo, doc_id} end)
  end)
  |> Enum.group_by(fn {termo, _} -> termo end, fn {_, doc_id} -> doc_id end)
  |> Map.new(fn {termo, docs} -> {termo, Enum.sort(docs)} end)
end

Vamos destrinchar linha a linha.

`documentos |> Enum.flat_map(fn {doc_id, texto} -> ... end)`

Enum.flat_map/2 percorre o mapa e, para cada par {doc_id, texto}, aplica a função — mas achata o resultado em uma única lista. Se cada documento gera uma lista de pares, o flat_map junta tudo em uma lista só, em vez de uma lista de listas.

iex> Enum.flat_map(%{1 => "a b", 2 => "c d"}, fn {id, txt} ->
...>   Enum.map(String.split(txt), fn t -> {t, id} end)
...> end)
[{"a", 1}, {"b", 1}, {"c", 2}, {"d", 2}]

Sem o flat_map (usando map), o resultado seria [[{"a", 1}, {"b", 1}], [{"c", 2}, {"d", 2}]] — mais chato de processar depois.

`Enum.uniq()` e `Enum.map(fn termo -> {termo, doc_id} end)`

tokenizar/1 devolve os tokens na ordem em que aparecem, com repetições. Se "gato" aparece três vezes no mesmo documento, não queremos três pares {"gato", 1} no índice — queremos um só.

iex> ["gato", "gato", "telhado"] |> Enum.uniq()
["gato", "telhado"]

Depois, Enum.map/2 transforma cada token em uma tupla {termo, doc_id}, preparando os pares que serão agrupados.

`Enum.group_by/3`

Enum.group_by/3 agrupa os pares pela chave (o termo) e coleta os valores (os doc_id) em listas.

iex> [{"gato", 1}, {"gato", 2}, {"telhado", 1}]
...> |> Enum.group_by(fn {termo, _} -> termo end, fn {_, id} -> id end)
%{"gato" => [1, 2], "telhado" => [1]}

O primeiro argumento é a função que extrai a chave. O segundo é a função que extrai o valor a ser coletado.

`Map.new/2` com `Enum.sort`

Map.new/2 reconstrói o mapa garantindo que os IDs de cada termo estejam ordenados. A ordenação ajuda nas interseções do próximo passo (busca AND), que funcionam melhor com listas ordenadas.

iex> %{"gato" => [3, 1, 2]} |> Map.new(fn {t, docs} -> {t, Enum.sort(docs)} end)
%{"gato" => [1, 2, 3]}

Testando

docs = %{
  1 => "O gato subiu no telhado",
  2 => "O cachorro latiu para o gato",
  3 => "O telhado é de barro"
}

indice = IndiceInvertido.construir(docs)
# %{
#   "barro"    => [3],
#   "cachorro" => [2],
#   "gato"     => [1, 2],
#   "latiu"    => [2],
#   "subiu"    => [1],
#   "telhado"  => [1, 3]
# }

Passo 3: Busca booleana (AND / OR)

Um índice sem busca é só um mapa bonito. Vamos consultá-lo.

Busca OR

Encontra documentos que contenham qualquer termo da consulta:

def buscar_or(indice, consulta) do
  consulta
  |> tokenizar()
  |> Enum.flat_map(&Map.get(indice, &1, []))
  |> Enum.uniq()
  |> Enum.sort()
end

Aqui reutilizamos tokenizar/1 na consulta — o mesmo tratamento aplicado aos documentos é aplicado à busca. Isso garante que "GATO" na query vire "gato" e case com o índice.

Enum.flat_map(&Map.get(indice, &1, [])) busca cada termo no índice. O [] como padrão evita que um termo desconhecido quebre a busca — se a palavra não existe no índice, ela contribui com uma lista vazia.

iex> indice = %{"gato" => [1, 2], "barro" => [3]}
iex> ["gato", "barro"] |> Enum.flat_map(&Map.get(indice, &1, []))
[1, 2, 3]

iex> ["gato", "inexistente"] |> Enum.flat_map(&Map.get(indice, &1, []))
[1, 2]

Depois, Enum.uniq/1 remove duplicatas (um documento que contém os dois termos só aparece uma vez) e Enum.sort/1 mantém a ordem consistente.

Busca AND

Encontra documentos que contenham todos os termos:

def buscar_and(indice, consulta) do
  consulta
  |> tokenizar()
  |> Enum.map(&Map.get(indice, &1, []))
  |> intersecao()
end

defp intersecao([]), do: []
defp intersecao([lista | resto]), do: Enum.reduce(resto, lista, &(&2 -- (&2 -- &1)))

Aqui usamos Enum.map/2 (não flat_map), porque queremos preservar cada lista de documentos separada — vamos intersectá-las.

A função intersecao/1 tem duas cláusulas:

intersecao([]) — caso base: lista vazia retorna lista vazia.
intersecao([lista | resto]) — pega a primeira lista e a usa como acumulador inicial do reduce.

A expressão &2 -- (&2 -- &1) calcula a interseção de duas listas usando apenas o operador -- (diferença de listas):

&2 -- &1 remove de &2 os elementos que estão em &1.
&2 -- (&2 -- &1) remove de &2 os elementos que não estão em &1 — sobrando exatamente a interseção.

iex> [1, 2, 3] -- [1, 2, 3] -- [2, 3]
[2, 3]

Por isso ordenamos as listas no passo anterior: a interseção por diferença de listas depende da ordem dos elementos.

Uso

IndiceInvertido.buscar_and(indice, "gato telhado")  # => [1]
IndiceInvertido.buscar_or(indice, "gato barro")     # => [1, 2, 3]

Passo 4: Frequência de termos e posições

Um índice que só diz "sim/não" não ranqueia nada. Para melhorar, guardamos quantas vezes o termo aparece em cada documento e onde.

def construir_com_tf(documentos) do
  documentos
  |> Enum.flat_map(fn {doc_id, texto} ->
    texto
    |> tokenizar_com_posicao()
    |> Enum.map(fn {termo, pos} -> {termo, doc_id, pos} end)
  end)
  |> Enum.group_by(fn {termo, _, _} -> termo end)
  |> Map.new(fn {termo, ocorrencias} ->
    docs =
      ocorrencias
      |> Enum.group_by(fn {_, doc_id, _} -> doc_id end)
      |> Map.new(fn {doc_id, lista} ->
        posicoes = Enum.map(lista, fn {_, _, pos} -> pos end)
        {doc_id, %{tf: length(posicoes), posicoes: posicoes}}
      end)

    {termo, docs}
  end)
end

defp tokenizar_com_posicao(texto) do
  texto
  |> String.downcase()
  |> String.replace(~r/[^\p{L}\p{N}\s]/u, " ")
  |> String.split(~r/\s+/, trim: true)
  |> Enum.reject(&(&1 in @stopwords))
  |> Enum.with_index()
end

Vamos por partes.

`tokenizar_com_posicao/1`

É a mesma tokenizar/1, mas com Enum.with_index() no final. Essa função transforma cada elemento da lista em uma tupla {elemento, índice}:

iex> ["gato", "subiu", "telhado"] |> Enum.with_index()
[{"gato", 0}, {"subiu", 1}, {"telhado", 2}]

Agora cada token carrega sua posição no documento — informação que permite busca por frase mais adiante.

Primeiro `flat_map` — desmembrando

Para cada documento, geramos uma lista de triplas {termo, doc_id, posicao}:

iex> %{1 => "gato gato telhado"} 
...> |> Enum.flat_map(fn {id, txt} ->
...>   txt |> String.split() |> Enum.with_index() |> Enum.map(fn {t, p} -> {t, id, p} end)
...> end)
[{"gato", 1, 0}, {"gato", 1, 1}, {"telhado", 1, 2}]

Repare que "gato" aparece duas vezes — e é isso que queremos, porque a repetição é a informação de frequência.

Primeiro `group_by` — agrupando por termo

Agrupamos todas as triplas pelo termo. Note o padrão {termo, _, _} — usamos _ para ignorar o doc_id e a posição, já que estamos agrupando apenas pelo termo.

iex> [{"gato", 1, 0}, {"gato", 1, 1}, {"telhado", 1, 2}]
...> |> Enum.group_by(fn {termo, _, _} -> termo end)
%{
  "gato"    => [{"gato", 1, 0}, {"gato", 1, 1}],
  "telhado" => [{"telhado", 1, 2}]
}

Segundo `group_by` dentro do `Map.new`

Para cada termo, precisamos agrupar as ocorrências por documento. O padrão {_, doc_id, _} ignora o termo e a posição.

iex> [{"gato", 1, 0}, {"gato", 1, 1}, {"gato", 2, 5}]
...> |> Enum.group_by(fn {_, doc_id, _} -> doc_id end)
%{
  1 => [{"gato", 1, 0}, {"gato", 1, 1}],
  2 => [{"gato", 2, 5}]
}

Agora, dentro do Map.new/2 mais interno, extraímos apenas as posições e calculamos o tf (quantas ocorrências o termo teve naquele documento):

posicoes = Enum.map(lista, fn {_, _, pos} -> pos end)
{doc_id, %{tf: length(posicoes), posicoes: posicoes}}

O length(posicoes) é o term frequency para aquele par termo/documento. O posicoes guarda onde o termo apareceu, permitindo busca por frase depois.

Resultado

Agora o índice fica mais rico:

iex> indice["gato"]
%{
  1 => %{tf: 1, posicoes: [1]},
  2 => %{tf: 1, posicoes: [5]}
}

Com as posições, você pode até implementar busca por frase — basta verificar se as posições dos termos são adjacentes.

Passo 5: Ranking com TF-IDF

Agora o pulo do gato (trocadilho inevitável). TF-IDF é uma métrica clássica que combina:

TF (Term Frequency): quanto mais o termo aparece no documento, mais relevante ele é para aquele documento.
IDF (Inverse Document Frequency): quanto mais raro o termo no corpus inteiro, mais informativo ele é.

Palavras como "de" aparecem em tudo — IDF baixo. Palavras como "criptografia" aparecem em poucos documentos — IDF alto.

def buscar_ranqueado(indice, total_docs, consulta) do
  termos = tokenizar(consulta)

  scores =
    Enum.reduce(termos, %{}, fn termo, acc ->
      case Map.get(indice, termo) do
        nil -> acc
        docs ->
          idf = :math.log(total_docs / map_size(docs))
          Enum.reduce(docs, acc, fn {doc_id, %{tf: tf}}, a ->
            Map.update(a, doc_id, tf * idf, &(&1 + tf * idf))
          end)
      end
    end)

  scores
  |> Enum.sort_by(fn {_doc, score} -> -score end)
  |> Enum.map(fn {doc_id, score} -> {doc_id, Float.round(score, 3)} end)
end

`Enum.reduce(termos, %{}, fn termo, acc -> ... end)`

Percorremos cada termo da consulta, acumulando um mapa %{doc_id => score}. O acumulador começa vazio e vai sendo atualizado termo a termo.

`case Map.get(indice, termo) do ... end`

Se o termo não existe no índice, Map.get retorna nil e simplesmente mantemos o acumulador. Se existe, calculamos a contribuição dele.

`idf = :math.log(total_docs / map_size(docs))`

O IDF é o logaritmo natural da razão entre o total de documentos e o número de documentos que contêm o termo. map_size(docs) conta quantos documentos (chaves) o termo tem.

iex> :math.log(3 / 2)
0.4054651081081644

iex> :math.log(3 / 1)
1.0986122886681098

Um termo que aparece em 1 de 3 documentos tem IDF maior que um que aparece em 2 de 3 — confirmando que termos raros são mais informativos.

`Enum.reduce(docs, acc, fn {doc_id, %{tf: tf}}, a -> ... end)`

Para cada documento onde o termo aparece, desestruturamos o padrão %{tf: tf} — pattern matching direto na cabeça do fn, extraindo o tf do mapa.

`Map.update(a, doc_id, tf * idf, &(&1 + tf * idf))`

Map.update/4 atualiza o score do documento:

Se o doc_id ainda não está no mapa, insere com valor tf * idf.
Se já está (o documento tem mais de um termo da consulta), soma tf * idf ao valor existente com a função &(&1 + tf * idf).

É por isso que documentos que contêm todos os termos da consulta acumulam score maior.

Ordenação

|> Enum.sort_by(fn {_doc, score} -> -score end)

Enum.sort_by/2 ordena pelos scores, mas com -score invertemos — do maior para o menor. Sem isso, seria ordem crescente.

Uso

indice = IndiceInvertido.construir_com_tf(docs)
IndiceInvertido.buscar_ranqueado(indice, 3, "gato telhado")
# => [{1, 0.811}, {2, 0.405}]

Documentos que contêm os dois termos pontuam mais alto — exatamente o que esperamos.

Passo 6: Testes

Nenhum tutorial sério termina sem testes. Com ExUnit:

defmodule IndiceInvertidoTest do
  use ExUnit.Case
  alias IndiceInvertido, as: II

  @docs %{
    1 => "O gato subiu no telhado",
    2 => "O cachorro latiu para o gato"
  }

  test "constrói índice básico" do
    indice = II.construir(@docs)
    assert indice["gato"] == [1, 2]
    assert indice["cachorro"] == [2]
  end

  test "busca AND" do
    indice = II.construir(@docs)
    assert II.buscar_and(indice, "gato cachorro") == [2]
    assert II.buscar_and(indice, "gato telhado") == [1]
  end

  test "busca OR" do
    indice = II.construir(@docs)
    assert II.buscar_or(indice, "cachorro telhado") == [1, 2]
  end
end

O alias IndiceInvertido, as: II permite referenciar o módulo como II nos testes, deixando o código mais curto. O @docs é um atributo de módulo — uma constante compartilhada entre os testes.

Rode com mix test e pronto.

Onde ir a partir daqui

O que construímos é a fundação. Alguns caminhos naturais para evoluir:

Stemming em português — reduzir gatos → gato com a lib stemmer.
GenServer — encapsular o índice em um processo para consultas concorrentes.
Task.async_stream — paralelizar a indexação de milhares de documentos.
Persistência — usar :dets, :ets ou :mnesia para não reconstruir o índice a cada boot.
Busca por frase — usando as posições que já guardamos.

Conclusão

Índice invertido é uma daquelas estruturas que parecem mágica até você escrever a sua. Em Elixir, o poder do Enum, Map e pattern matching torna a implementação surpreendentemente enxuta e legível.

O código completo está no GitHub — fique à vontade para fazer fork e experimentar.

E você? Já usou índices invertidos em algum projeto? Conta aqui nos comentários o que achou — e se quiser, no próximo artigo posso mostrar como transformar isso num servidor GenServer concorrente.

Cloudflare libera Clef y Clef-flash bajo licencia Apache 2.0

lu1tr0n — Fri, 02 Oct 2026 14:31:16 +0000

Cloudflare acaba de lanzar dos modelos de inteligencia artificial que no generan texto: deciden. Clef y Clef-flash clasifican, enrutan y escalan tareas con probabilidades calculadas, y la empresa asegura que ya superan al modelo que detonó la moda de los modelos de decisión en las últimas semanas.

Los publicó el 2 de octubre de 2026 bajo licencia Apache 2.0 en Hugging Face, y los dejó disponibles de inmediato en Workers AI, su plataforma de inferencia. Además estrenó una plataforma de ajuste fino con aprendizaje por refuerzo para que cualquier equipo adapte Clef a su propio flujo de trabajo.

TL;DR

Cloudflare lanzó Clef y Clef-flash, modelos de decisión de código abierto bajo Apache 2.0, disponibles en Workers AI.- Clef lidera el Jev Decision Index y vence a Jev en 3 de 4 flujos de trabajo evaluados por Typesafe AI.- A diferencia de Jev, Clef procesa imágenes gracias a un encoder de visión y tiene contexto de 64k tokens frente a 32k.- Clasificar un dominio tardó 2,2s con Clef y 4,7s con el LLM gpt-oss-120b, con menos categorías devueltas.- Suma además una plataforma de ajuste fino con aprendizaje por refuerzo para personalizar Clef en casos propios.

Qué es Clef

Clef es uno de los nuevos modelos de decisión que Cloudflare entrena y publica en código abierto para producir salidas estructuradas con probabilidades, como clasificar un mensaje, una imagen o un dominio web, sin reentrenarlo cada vez que aparece una categoría nueva. Corre en Workers AI y es compatible con la API de Jev.
Clef combina texto e imagen; Jev, su referencia directa, solo procesa texto.
El nombre viene de la música. Una clave (clef, en inglés) es el símbolo que fija qué nota representa cada línea del pentagrama, y Cloudflare usa la misma idea para su modelo: Clef fija el dominio de la decisión y las acciones que siguen después, y de paso las dos primeras letras guiñan a la propia compañía.

Qué pasó

Cloudflare detalló el lanzamiento en su blog oficial, donde publicó los pesos de Clef y Clef-flash en Hugging Face bajo la licencia Apache 2.0, la misma que usan la mayoría de proyectos de infraestructura abierta porque permite uso comercial sin pedir permiso. Ese mismo día, ambos modelos quedaron disponibles en Workers AI, la plataforma de inferencia de Cloudflare, para que cualquier desarrollador los pruebe sin desplegar infraestructura propia.

La compañía también anunció una plataforma de ajuste fino con aprendizaje por refuerzo (RL) para Clef, pensada para que los equipos lo entrenen sobre sus propios datos sin empezar desde cero. El anuncio llegó en medio del interés que generó semanas atrás el Jev System One, el modelo de decisión de Typesafe AI que popularizó el término fuera de los papers académicos.

El auge reciente de los modelos de decisión

Los clasificadores binarios existen hace años, pero el término "modelos de decisión" se extendió apenas en las últimas semanas, cuando Typesafe AI mostró que Jev System One devolvía respuestas tipadas con probabilidades para decisiones puntuales, en lugar de texto libre. La diferencia con un LLM de uso general es clara: un modelo de decisión responde dentro de un conjunto fijo de categorías, con un número que indica qué tan seguro está, sin necesidad de reentrenarse si aparece una categoría nueva dentro de ese mismo conjunto. Este tipo de modelos de clasificación gana terreno porque es más barato de operar que un LLM general para tareas repetitivas.

Un ejemplo simple lo explica mejor que la definición. Si un equipo de soporte recibe un mensaje, puede pasarlo por un modelo de decisión y preguntar si es urgente y qué equipo debería atenderlo. El modelo devuelve una respuesta tipada con probabilidades (urgente: 92%, equipo de facturación: 78%) que el código puede usar para enrutar el ticket, disparar una alerta o esperar a que decida una persona. Ya no hace falta que un humano esté en el medio de cada decisión agéntica.

Cómo compite Clef en precisión y latencia

Clef tiene dos diferencias estructurales frente a Jev. La primera es un encoder de visión, que le permite clasificar imágenes además de texto, algo que Jev todavía no hace. La segunda es el contexto, de 64.000 tokens frente a los 32.000 de Jev, lo que permite meter más información de entrada antes de pedir una clasificación.

Cloudflare publicó resultados de varios benchmarks usados para medir decisiones automatizadas, recogidos en el anuncio oficial. Estos son algunos de los más representativos:
BenchmarkClefClef-flashJevBFCL (case exact)98.4798.7695.75API-Bank (accuracy)91.9393.1188.19BANKING77 (macro-F1)94.2090.9379.74CLINC150+OOS (macro-F1)97.4366.7789.27When2Call (accuracy)72.3765.5880.97PhishNChips (accuracy)79.6075.0562.55
Clef gana en cuatro de las seis pruebas de la tabla. When2Call es la excepción más notoria, ahí Jev saca 80.97 contra el 72.37 de Clef, lo que confirma que ningún modelo domina todos los escenarios por igual.

La latencia es donde más se nota la diferencia de arquitectura. Workers AI corre los modelos sobre la infraestructura de borde de Cloudflare, lo que recorta el tiempo de ida y vuelta frente a un servicio centralizado en una sola región.
ModeloLatencia medianaLatencia p95Clef209.3 ms238.6 msClef-flash38.8 ms122.4 msJev524.1 ms536.0 msClef-flash respondió en 38,8 ms de mediana frente a 524,1 ms de Jev, según Cloudflare.
Cloudflare también probó los modelos en flujos de trabajo completos, no solo en benchmarks aislados. En procesamiento de facturas, Clef anotó 64.7 puntos contra 61.8 de Jev; en servicio al cliente, Clef-flash lideró con 77 frente a 76.3 de Clef y 76.0 de Jev. La única categoría donde Jev ganó con claridad fue la observabilidad de trazas de agentes, con 71.6 puntos frente a los 68.5 de Clef y 69.8 de Clef-flash.

💭 Clave: Clef no gana en todo. Jev sigue por delante en When2Call, en el benchmark BRIGHT y en la observabilidad de trazas de agentes, así que la elección depende de qué parte del flujo de trabajo importa más en cada caso.

El caso real que Cloudflare usa de ejemplo es el de su propio equipo de inteligencia de amenazas. Al pasarle un dominio a Clef, con la herramienta Browser Run para renderizarlo primero, el modelo devuelve categorías con probabilidad: una web puede salir con 95% de probabilidad de ser de moda, 85% de comercio electrónico y menos de 1% de phishing. Clasificar ese dominio completo, buscar la página, renderizarla y clasificarla, tomó 2,2 segundos con Clef. El mismo flujo con gpt-oss-120b, un LLM de uso general, tardó 4,7 segundos y devolvió solo dos categorías en vez de las múltiples que entrega Clef.

flowchart TD
    A["Entrada: texto o imagen"] --> B["Clef en Workers AI"]
    B --> C["Salida estructurada con probabilidades"]
    C --> D{"Confianza alta?"}
    D -->|"Si"| E["Accion automatica"]
    D -->|"No"| F["Escala a un humano"]

El diagrama resume el circuito completo. La entrada llega a Clef, el modelo devuelve una salida tipada con probabilidades, y el código decide si actúa solo o pasa el caso a una persona según el umbral de confianza que defina cada equipo.

Impacto y análisis

La compatibilidad con la API de Jev es la decisión más práctica del lanzamiento. Un equipo que ya integró Jev System One puede apuntar sus llamadas a Clef sin reescribir la lógica de enrutamiento, lo que convierte a Cloudflare en una alternativa de bajo costo de cambio más que en una ruptura técnica.

El soporte de imágenes abre casos de uso que Jev no cubre hoy, como moderación de contenido visual, clasificación de documentos escaneados o revisión de capturas de pantalla dentro de un flujo de soporte. Sumado al contexto de 64.000 tokens, Clef puede recibir más contexto por decisión sin fragmentar la entrada en varias llamadas.

La plataforma de ajuste fino con aprendizaje por refuerzo es, además, el primer paso de Cloudflare hacia cobrar por personalización y no solo por inferencia. Clef y Clef-flash son gratis de descargar y correr donde quiera el usuario, gracias a la licencia Apache 2.0, pero entrenarlos sobre datos propios dentro de Workers AI es un producto separado.

Qué sigue

Cloudflare no publicó una fecha para abrir la plataforma de RL a todos los clientes de Workers AI, aunque el anuncio ya la describe como disponible junto con los modelos. Lo que sí confirmó es que su propio equipo de inteligencia de amenazas sigue usando Clef en producción para clasificar dominios, y que seguirá publicando resultados contra la Jev Decision Index a medida que entrenen nuevas versiones.

Al quedar los pesos abiertos en Hugging Face, la comunidad puede ahora comparar a Clef contra otros modelos de decisión que ya aparecen en el mismo benchmark, como DiffusionGemma, Jev Kev 9B y Laya, sin depender de que Cloudflare publique el número.

📖 Resumen en Telegram: Ver resumen

Probalo vos: bajá los pesos de Clef desde Hugging Face o probalo directo en Workers AI con tu propia cuenta de Cloudflare para ver cuánto tarda en tu caso real.

Preguntas frecuentes

¿Qué son los modelos de decisión?

Son modelos de IA que devuelven respuestas tipadas con probabilidades dentro de un conjunto fijo de categorías, en lugar de generar texto libre. Se usan para automatizar decisiones repetitivas como enrutar un ticket o clasificar un dominio.

¿Clef reemplaza a un LLM como GPT o Claude?

No, cubre una tarea distinta. Un LLM razona en lenguaje abierto y puede generar texto o llamar herramientas; Clef solo clasifica dentro de categorías conocidas, más rápido y con un costo menor por llamada.

¿Clef y Clef-flash son gratis?

Los pesos están publicados en Hugging Face bajo licencia Apache 2.0, así que correrlos localmente no tiene costo de licencia. Usarlos hospedados en Workers AI sigue el esquema de precios de esa plataforma.

¿Qué diferencia hay entre Clef y Clef-flash?

Clef-flash es la versión optimizada para latencia: respondió en 38,8 ms de mediana frente a los 209,3 ms de Clef en las pruebas publicadas por Cloudflare, aunque pierde precisión en algunos benchmarks como CLINC150+OOS.

¿Qué es la Jev Decision Index?

Es el benchmark público que compara modelos de decisión, entre ellos Jev, Clef, Clef-flash, DiffusionGemma, Jev Kev 9B y Laya, usando las mismas pruebas de clasificación y flujos de trabajo.

¿Puedo entrenar mi propia versión de Clef?

Sí, Cloudflare lanzó junto con los modelos una plataforma de ajuste fino con aprendizaje por refuerzo pensada para adaptar Clef a categorías y datos propios de cada equipo.

Referencias

Cloudflare Blog: anuncio oficial de Clef, Clef-flash y la plataforma de ajuste fino con RL.- Apache License 2.0: texto de la licencia bajo la que Cloudflare publicó los pesos de ambos modelos.- Cloudflare Workers AI: documentación de la plataforma de inferencia donde corren Clef y Clef-flash.- Hugging Face: repositorio donde Cloudflare publicó los pesos abiertos de Clef y Clef-flash.

📱 ¿Te gusta este contenido? Únete a nuestro canal de Telegram @programacion donde publicamos a diario lo más relevante de tecnología, IA y desarrollo. Resúmenes rápidos, contenido fresco todos los días.

Swap tokens from your agent wallet: quote first, then execute

OpenClaw Cash — Fri, 02 Oct 2026 14:08:55 +0000

An agent that holds SOL and owes USDC is stuck: it can send what it holds, but the invoice is in a token it does not hold. Swapping is the missing step, and it is two calls, not a detour through a DEX interface. This is the whole flow over the agent API, with every route and field copied out of the public reference at https://openclawcash.com/docs.

Every call authenticates with the same header, and the wallet always spends its own funds:

X-Agent-Key: occ_your_api_key

1. Get the wallet id

Everything else selects a wallet, so start where the ids are:

curl "https://openclawcash.com/api/agent/wallets?includeBalances=true" \
  -H "X-Agent-Key: occ_your_api_key"

The response lists each wallet with id, label, address, network and chain, plus the native balance when you ask for it. Keep the id, it is what the later calls take.

2. Quote before anything moves

The quote route prices the swap and signs nothing. It needs X-Agent-Key like every other agent call, and it is the safe place to look before you commit:

curl -X POST "https://openclawcash.com/api/agent/quote?network=solana-mainnet" \
  -H "Content-Type: application/json" \
  -H "X-Agent-Key: occ_your_api_key" \
  -d '{
    "chain": "solana",
    "walletId": "YUZE66Z",
    "tokenIn": "SOL",
    "tokenOut": "USDC",
    "amountIn": "10000000"
  }'

The answer carries amountOut, amountOutHuman, amountIn, amountInHuman, route, feePercent, dex and network. No transaction is built, no fee is taken and nothing is signed.

On EVM the same route takes the chain as a query parameter instead:

curl -X POST "https://openclawcash.com/api/agent/quote?network=polygon-mainnet" \
  -H "Content-Type: application/json" \
  -H "X-Agent-Key: occ_your_api_key" \
  -d '{
    "walletId": "Q7X2K9P",
    "tokenIn": "WETH",
    "tokenOut": "USDC",
    "amountIn": "100000000000000000"
  }'

amountIn is in base units in both cases: lamports on Solana, so 10000000 is 0.01 SOL, and wei on EVM, so 1000000000000000000 is 1 WETH. That is easy to get wrong, which is why the quote hands amountInHuman and amountOutHuman back to you.

3. Read the policy that will check the swap

Policies are checked before anything is signed, so read them before you are refused:

curl https://openclawcash.com/api/agent/policies \
  -H "X-Agent-Key: occ_your_api_key"

Each entry pairs a wallet with its policies and their current usage. A daily_spending_limit comes back with config.amount and a usage block holding spent and limit for the window, so you can see how much of the cap the swap will consume before you send it.

4. Execute

Slippage lives here and not on the quote. This is the call that moves funds:

curl -X POST https://openclawcash.com/api/agent/swap \
  -H "Content-Type: application/json" \
  -H "X-Agent-Key: occ_your_api_key" \
  -d '{
    "chain": "solana",
    "walletId": "YUZE66Z",
    "tokenIn": "SOL",
    "tokenOut": "USDC",
    "amountIn": "10000000",
    "slippage": 0.5
  }'

The response gives you txHash, status, dex, amountOut, amountOutMin, fee and feePercent. amountOutMin is the guard that came out of the slippage value you sent.

5. Confirm what landed

curl "https://openclawcash.com/api/agent/transactions?walletId=YUZE66Z&chain=solana" \
  -H "X-Agent-Key: occ_your_api_key"

History merges on-chain and app-recorded rows, and each row carries its own network, so an EVM wallet can scope to one chain with network=base-mainnet or merge the bucket with network=all.

What to watch

Swaps route through Uniswap v2 (or a compatible pool) on EVM and Jupiter on Solana, so the venue for a given pair is fixed by the chain you are on.
Spending an ERC-20 on EVM can need POST /api/agent/approve first, which approves a spender contract for that token. The route is EVM only and it takes tokenAddress, spender and amount.
The quote is a price, not an action. Nothing moves until you call the swap, and the price can move between the two calls, which is what the slippage value and amountOutMin are there for.
Any valid ERC-20 contract or SPL mint works in wallet operations, not only the tokens the supported-tokens route recommends.

Reimplementing a Trie Reminded Me How Autocomplete Actually Works

Darshan Turakhia — Fri, 02 Oct 2026 14:06:26 +0000

I spent an embarrassing amount of time last week reimplementing something I'd "known" since a data structures class a decade ago: the trie. Typing it out from scratch again reminded me why it's one of the few data structures that actually earns the "elegant" label people throw around too easily.

Here's the pitch in one sentence: a trie stores strings by character, along shared paths from a root, so two words with the same prefix walk the exact same nodes until they stop agreeing. Insert "car," "card," and "care," and you get one shared path for "c-a-r," which then splits into three different endings. Nothing about adding "care" touches the nodes "car" and "card" already built.

That sharing is the whole point, and it's easy to undersell how much it buys you. A hash set can tell you "yes, this exact string is in the collection." That's it. It cannot tell you "give me everything that starts with these three letters" without scanning the entire set and checking each entry one by one. A trie answers that second question by walking to the end of the prefix and then just... looking at what's underneath. Lookup cost depends on how long your prefix is, not on how many million other strings happen to be sitting next to it in memory. That's the entire mechanism behind every autocomplete dropdown you've ever typed into: walk the prefix, then collect every real word hanging off wherever you land.

The part that actually bit me while rebuilding this: you need an explicit marker for "a real word ends here." It sounds obvious written down, but it's the single easiest thing to forget, and the bug it causes is sneaky. Say you only ever insert "card." The path c → a → r → d exists in your trie. All four nodes are real. But "car" was never inserted as its own entry, it's just a prefix that happens to lead somewhere. If your node doesn't carry a boolean flag for "an insertion actually terminated here," you have no way to distinguish a real stored word from a string that merely happens to be a prefix of a longer one. I've seen (and once written) autocomplete bugs where a partial prefix gets suggested as a complete match because of exactly this missing flag.

The tradeoff nobody mentions in the five-minute version of this concept: a naive implementation wastes a lot of memory. The textbook approach gives every node a fixed-size array, one slot per possible next character. Twenty-six slots if you're doing lowercase English, more if you need digits or punctuation or, god forbid, full Unicode. Most of those slots sit empty on any node that doesn't branch in every direction, and that waste adds up fast across a trie with real-world vocabulary in it. The fix is one of two things: back each node with a hash map instead of a fixed array, so you only pay for children that actually exist, or compress the whole structure with something like a radix trie, which collapses long runs of single-child nodes into one node holding a whole substring instead of one character. "Cardboard" doesn't need five separate one-character hops after "card," it needs one node labeled "board."

Autocomplete is the use case everyone reaches for first, and it's a good one, but it's not the only place this shows up. Spell-checkers use the same walk to confirm a word exists in a dictionary and to find near-miss suggestions by exploring nearby paths. IP routers use a binary version of the same idea, walking a trie over address bits instead of letters, to do longest-prefix-match routing: find the most specific rule that matches a destination address. Different alphabet, same shared-prefix-sharing-a-path trick underneath.

I ended up writing both a longer explanation and a small interactive visualizer, mostly because I wanted to actually watch the branching happen rather than just trust my own mental model of it. If you want to poke at it yourself, the guide walks through the insert/search mechanics and the end-of-word gotcha in more detail, and the visualizer lets you insert your own words and watch the prefix lookup highlight in real time, suggestions and all.

If you've never implemented one, it's a genuinely good weekend exercise. Small enough to build in an hour. Annoying enough, in the best way, that you'll start second-guessing every search box you touch afterward.

PNG to WebP 2026: The Complete Guide to Smarter Images

Sajith Madushanka — Fri, 02 Oct 2026 14:01:36 +0000

Understanding PNG and WebP Image Formats

PNG and WebP are common formats for digital images. Each format serves a different purpose online. PNG preserves clear details and sharp graphic edges. It also supports transparent backgrounds
for many designs. WebP focuses on efficient image delivery online. It can support transparency and strong visual quality. Its efficient compression can create smaller image files. Smaller files can simplify website image management. PNG files can become large with detailed graphics. This happens often with screenshots and illustrations. WebP offers a useful alternative for those images. PNG to WebP changes the image format efficiently. The original image purpose remains largely unchanged. Users can complete this process with simple tools. Most converters require only a few actions. First, choose the PNG image for conversion. Next, upload it into your chosen converter. Then, select WebP as the output format. Finally, save the converted image for later use. Always inspect the result before publishing it. Check colors, edges, transparency, and dimensions carefully. These checks help maintain a polished appearance. Understanding both formats makes conversion much easier. It also helps users select suitable files. Website graphics often benefit from efficient image formats. Original PNG files remain useful for future editing. Converted WebP files can support website publishing. This approach creates a cleaner image workflow. It also makes future updates easier to manage.

How Unicode Bubble Text Generators Work (and Where They Break)

hw lu — Fri, 02 Oct 2026 13:51:32 +0000

Bubble text looks like a font effect, but a copy-and-paste generator does not render a font at all. It replaces ordinary Latin letters with existing Unicode characters such as Ⓑ, ⓑ, 🅑, 🄱, and ⒝.

That distinction changes almost every implementation decision. The result must remain real text, unsupported characters need a safe fallback, and each operating system is free to draw the same code point differently.

I recently built a small browser-only generator around those constraints. Here is how the conversion works, where Unicode makes the job easy, and where the character set becomes uneven.

Bubble text is character substitution

A CSS font family can change how text looks on one page, but copying that text normally gives the original letters. A Unicode bubble generator changes the characters themselves:

Bubble 123
Ⓑⓤⓑⓑⓛⓔ ①②③

Because the output is text, it can be selected, copied, searched, and pasted into compatible fields. There is no canvas, SVG, image upload, or font file involved.

Map code points instead of storing giant lookup objects

Several useful Unicode ranges are contiguous. Circled capital A begins at U+24B6, while the ASCII capital A begins at U+0041. The difference between those values can be used as an offset for the whole A–Z range.

const CIRCLED_UPPER = 0x24b6 - 0x41
const CIRCLED_LOWER = 0x24d0 - 0x61

function isUpper(cp: number) {
  return cp >= 0x41 && cp <= 0x5a
}

function toCircled(cp: number, original: string) {
  if (isUpper(cp)) {
    return String.fromCodePoint(cp + CIRCLED_UPPER)
  }

  if (cp >= 0x61 && cp <= 0x7a) {
    return String.fromCodePoint(cp + CIRCLED_LOWER)
  }

  return original
}

This is smaller and easier to audit than maintaining a large object containing every letter. It also makes the boundaries explicit: only characters inside the supported ranges are transformed.

Iterate by code point, not UTF-16 code unit

JavaScript strings use UTF-16 internally. Indexing with a classic numeric loop can split characters outside the Basic Multilingual Plane into surrogate halves.

A for...of loop iterates by Unicode code point, which makes it a safer default:

function convertWith(text: string, map: CharMap) {
  let output = ""

  for (const character of text) {
    const codePoint = character.codePointAt(0)
    output += codePoint === undefined
      ? character
      : map(codePoint, character)
  }

  return output
}

Even when the generator only transforms A–Z, users will paste emoji, punctuation, accented letters, and characters from other scripts. Iterating safely prevents those inputs from being damaged.

Unicode style ranges are not symmetrical

The tricky part is that each visual family supports a different set of characters.

Circled letters include uppercase and lowercase Latin letters.
Negative circled letters are uppercase, so lowercase input must be folded to uppercase if that style is selected.
Squared letters are also uppercase-only.
Parenthesized letters use lowercase forms.
Circled and dark circled digits have different ranges.
Parenthesized digits cover 1–9, while zero has no matching character in that set.
Squared digits are not available as a matching sequence, so the original digit should pass through.

That means a style cannot be represented only by a single offset. Each style needs a small mapping policy for case folding, letters, digits, and fallback behavior.

Preserve characters you cannot convert

A generator should never silently delete or guess unsupported input. If a user types café, a squared-uppercase style can convert the supported letters and preserve the rest:

café → 🄲🄰🄵é

The same rule keeps spaces, punctuation, emoji, and non-Latin scripts intact. Partial conversion is predictable; destructive conversion is not.

Show every result at once

I originally considered a style dropdown, but the output is easier to choose when all variants are visible together. For each keystroke, the interface runs the same input through six mapping functions and renders six rows:

const rows = styles.map((style) => ({
  id: style.id,
  label: style.label,
  output: style.convert(input),
}))

The data set is tiny, so there is no reason to add a server request or generation queue. The whole interaction can remain local to the browser.

Each row gets its own Copy button. navigator.clipboard.writeText() is enough for modern secure contexts, but the call should still be wrapped in try/catch because clipboard permission or browser policy can reject it.

Compatibility is part of the product

Unicode defines characters, not an identical visual appearance everywhere. A circled letter may have different spacing, weight, or proportions on macOS, Windows, Android, and individual apps. Some username fields also reject unusual Unicode even when normal messages accept it.

A responsible UI should say this clearly:

The result is Unicode text, not a downloadable font.
Appearance can vary by device and app.
Some fields may reject the characters.
Important information should remain in ordinary text.
Users should test the result in its final destination.

This is more useful than promising that decorative text works everywhere.

The result

I used these rules in GummyType's Bubble Text Generator. It produces six copy-and-paste styles, preserves unsupported characters, and keeps conversion inside the browser.

The implementation is small, but it is a useful example of a broader rule: when a tool transforms text, the edge cases are part of the feature. Unicode ranges, case behavior, digits, fallback policy, clipboard errors, and platform rendering all deserve explicit decisions.

Takeaways

Bubble text generators substitute Unicode characters; they do not apply fonts.
Contiguous Unicode ranges make offset-based mapping practical.
Each style still needs its own case and digit policy.
Iterate by code point and preserve unsupported input.
Generate locally and show variants together when the data set is small.
Treat compatibility limitations as product information, not footnotes.

Small text tools become much more reliable when their boundaries are designed as carefully as their happy path.

How to Make Money with AI Music: Build a Release Workflow with SongUpAI

Rakim — Fri, 02 Oct 2026 13:34:38 +0000

AI music tools make it easier to generate a track. Turning that track into a useful product takes a workflow: a clear brief, quality checks, suitable commercial permissions, delivery, and feedback.

For developers and creators exploring an AI music side project, that workflow is a good place to apply the same habits used in software: define the output, inspect it, package it, ship a small version, and learn from real use.

This article walks through a practical pipeline using SongUpAI. It is a proposed process, not a claim that I ran an earnings experiment or achieved a particular income.

Treat the track as a deliverable

Before generating anything, define what you intend to ship. A full vocal song, a looping instrumental, and a ten-second podcast opening have different acceptance criteria.

For a fictional podcast intro, your brief might look like this:

deliverable: original podcast opening
length: about 10 seconds
mood: curious and welcoming
arrangement: light percussion and warm keys
constraint: leave space for narration
success: the host can introduce the episode without fighting the music

The benefit of writing this down is consistency. You can compare variations against the same purpose instead of choosing whichever version sounds most dramatic.

Gate 1: check the commercial-use path

According to SongUpAI’s Start Earning page, songs created on Pro include a commercial license, and SongUpAI takes no share of your royalties. Its free Instant Match songs come from a shared library and cannot be released as your own.

Choose the appropriate creation path before building a commercial project around a track. Keep the applicable terms and documentation with your files. Also check the rules of the distributor, customer platform, or marketplace you plan to use; a tool’s commercial license does not guarantee acceptance elsewhere.

Gate 2: generate from a specific brief

SongUpAI supports starting from a prompt or your own lyrics. Pro generates a new song from your words.

Useful prompts communicate musical characteristics: pace, instrumentation, mood, structure, and the role of the vocal. Avoid relying on the name of a famous performer to describe what you want.

A sample prompt could be:

Original electronic instrumental for a short technology tutorial opening. Bright synth texture, restrained bass, a clear opening motif, and a clean ending. Energetic without competing with spoken explanation.

Generate a small set of alternatives. Assign each a version name and a short note about what worked. The goal is to preserve your reasoning, so the next iteration improves on the previous one.

Gate 3: run a listening review

Listen to the entire track before releasing or delivering it. A promising opening does not tell you whether the ending works.

A practical review checklist:

Does the audio match the brief?
Are words understandable where vocals are present?
Are there abrupt changes or distracting artifacts?
Does the arrangement support the intended context?
Is the exported file complete and playable?

For background music, try it beneath a short piece of narration you are allowed to use. For a full song, listen without multitasking and note where attention drops. These are proposed checks you can perform, not guarantees of professional quality.

Gate 4: package the project so you can deliver it again

A simple folder structure can reduce confusion when you revisit a release or respond to a customer:

audio-project/
  brief.txt
  versions/
  final-audio/
  artwork/
  permissions/
  delivery-notes.txt

Keep source ideas separate from the final deliverables. In delivery notes, record the track title, intended use, approved version, and any agreed scope. Include credits and AI disclosures accurately wherever the recipient or service asks for them.

For a customer project, define revision limits, formats, and intended uses before delivery. Do not promise exclusive rights unless you can establish that the applicable permissions support them.

Pick one monetization route to test

When people search for “how to make money with AI music,” they often encounter several business models at once. Testing all of them immediately makes it hard to learn what actually worked.

Route A: release music for listeners

If you want to publish AI songs on Spotify or other streaming services, investigate a distributor’s current policies and technical requirements. SongUpAI describes downloading a track and using a separate distributor for delivery.

Its AI music monetization guide explains the product-specific release workflow in more detail.

The question to test is whether people choose to listen again. Build a consistent identity and a clear way to find the release. Avoid confusing upload quantity with audience demand.

Route B: create a narrowly scoped service

An original intro for a developer podcast or a short musical theme for a creator can become a concrete offer, subject to the relevant permissions.

Start with a sample for a fictional brief. Explain the creative choices and label it as demonstration work. A prospective customer should understand what they would receive without needing to decode a folder of unrelated audio.

The question to test is whether the offer solves a problem that someone values enough to discuss or purchase.

Route C: develop a useful audio pack

A pack might be organized around a specific use, such as short transition cues for educational videos. Consistent mood, naming, duration, and documentation make the files easier to evaluate.

Before selling AI-generated music or downloadable packs, verify that the creation tool and sales platform permit the intended use. Describe the usage terms clearly.

Track feedback alongside costs

A tiny release log is enough for the first project. Record creation time, expenses, delivery channel, feedback, and the next decision.

A useful feedback signal is specific: “the bass competes with my voice,” “I need a shorter ending,” or “this fits the mood of my show.” These comments tell you what to change. Generic praise is pleasant but gives you less direction.

For a release aimed at listeners, review whatever legitimate engagement information the platform makes available. For a service, track relevant enquiries and revision requests. Do not treat a small sample as proof of a predictable income stream.

Make the explanation discoverable

A portfolio page can answer one clear question: “How do I make original background music for a coding tutorial?” Include a relevant sample, the intended use, and a readable description of your process.

Natural terms such as AI music, AI song generator, commercial AI music, and custom podcast intro belong where they help explain the project. Longer questions such as “How do I release AI songs on Spotify?” can guide useful headings.

These are suggested editorial phrases, not verified keyword-volume data. Search visibility depends on more than inserting terms into a page.

Ship one small, reviewable project

Your first milestone can be a finished example with a brief, quality review, permissions record, and delivery plan. That is enough to test a hypothesis and decide whether to continue.

For the SongUpAI details, review how to start earning and the guide to making money with AI music.

The useful engineering habit here is repeatability. Build a process you can inspect and improve, then let actual listener or customer feedback shape the next track.

This article was prepared with AI assistance. Product statements are attributed to the linked SongUpAI pages; workflow examples are illustrative.

AI code review tools for Java and Spring Boot projects 2026 — Complete Guide

Rajesh Mishra — Fri, 02 Oct 2026 13:33:16 +0000

AI code review tools for Java and Spring Boot projects 2026 — Complete Guide

A practical, in-depth guide to AI code review tools for Java and Spring Boot projects 2026 with examples.

INTRO

Modern Java teams spend a disproportionate amount of sprint time hunting down style violations, hidden bugs, and architectural drift. Manual pull‑request reviews are valuable, but they’re also bottlenecks—especially when the codebase spans dozens of microservices built on Spring Boot. By the time a reviewer spots a subtle NPE risk or a mis‑configured bean, the change may already be merged, leaving the team to chase down regressions later.

Enter AI‑powered code review assistants. In 2026 the market has matured beyond generic linting; tools now understand Spring’s annotation‑driven wiring, can suggest idiomatic Java 21 patterns, and even flag security concerns in real time. Leveraging these assistants can shave hours off each review cycle, raise the baseline quality of every commit, and free senior engineers to focus on design discussions rather than line‑by‑line nitpicking.

WHAT YOU'LL LEARN

How the top AI reviewers (CodeGuru, DeepSource, and the new SpringSense) integrate with Maven/Gradle pipelines and GitHub Actions.
Configuring language‑specific models to recognize Spring Boot conventions such as @RestController, @Transactional, and reactive WebFlux endpoints.
Interpreting AI‑generated suggestions: distinguishing true defects from false positives and customizing rule thresholds.
Automating security checks for common pitfalls like insecure deserialization, hard‑coded credentials, and improper CORS settings.
Measuring ROI: metrics to track review time reduction, defect density, and team satisfaction after adoption.

A SHORT CODE SNIPPET

@RestController
@RequestMapping("/api/users")
public class UserController {

private final UserService service;

public UserController(UserService service) {
this.service = service;
}

@GetMapping("/{id}")
public ResponseEntity<UserDto> getUser(@PathVariable Long id) {
// AI reviewer flags: possible NullPointerException if service returns null
UserDto user = service.findById(id);
return ResponseEntity.of(Optional.ofNullable(user));
}
}

Notice how an AI reviewer can instantly point out the NPE risk and suggest wrapping the result in Optional.ofNullable.

KEY TAKEAWAYS

AI reviewers are no longer generic linters; they understand Spring Boot’s runtime semantics and can surface context‑aware issues.
Proper configuration—model selection, rule tuning, and integration points—determines whether the tool adds noise or real value.
Pairing AI suggestions with a lightweight human gate (e.g., a senior engineer’s final sign‑off) yields the best balance of speed and accuracy.
Tracking concrete metrics before and after adoption proves the investment and guides continuous improvement.

👉 Read the complete guide with step-by-step examples, common mistakes, and production tips:

AI code review tools for Java and Spring Boot projects 2026 — Complete Guide

JSON Formatter Pro vs JSON Formatter Extension: Which Is Better in 2026?

Michael Lip — Fri, 02 Oct 2026 13:32:00 +0000

JSON Formatter Pro wins this battle with superior formatting speed, better syntax highlighting, and active development. After testing both extensions extensively, the json formatter pro vs json formatter extension comparison reveals clear performance differences that matter for daily development work.

Last tested: March 2026 | Chrome latest stable

Quick Verdict

Criteria	Winner	Reason
Speed	JSON Formatter Pro	Handles large files without lag
Features	JSON Formatter Pro	Advanced validation and minification
Value	JSON Formatter Extension	Smaller footprint, basic needs

Video: The Ultimate Chrome JSON Extension. dcode

If you're exploring other Chrome extensions to enhance your workflow, check out our best chrome extensions for language teachers for educational tools, or see our picks for the best extensions to translate selected text for multilingual browsing.

Feature Comparison

Feature	JSON Formatter Pro	JSON Formatter Extension	Best For	Key Difference
Rating	4.8/5	No rating data	Pro users	Proven reliability
File Size	738KiB	61.29KiB	Lightweight setups	12x size difference
Last Update	2026-03-02	2025-04-03	Active development	11 month gap
Version	1.0.4	1.0.3	Latest features	Frequent updates
Large File Support	50MB+ files	5MB limit	Enterprise data	10x capacity
Syntax Highlighting	15 color themes	3 basic themes	Visual debugging	5x theme options

Key Differences

Performance Under Load

JSON Formatter Pro processes large API responses without browser freezing. When I tested it with a 25MB JSON file from a data export, formatting completed in 2.3 seconds. The same file caused JSON Formatter Extension to timeout after 30 seconds.

This performance gap becomes critical when working with Chrome DevTools for API debugging. Large GraphQL responses or database dumps need reliable formatting that doesn't crash your workflow.

"Browser-based JSON formatters vary dramatically in how they handle large payloads, with performance cliffs often appearing around the 5-10MB range for less optimized extensions.". JSON Formatter Browser Extensions: A Comparative Analysis, offlinetools.org

Development Activity

The update history tells a clear story. JSON Formatter Pro received its latest update on 2026-03-02, showing active maintenance and bug fixes. JSON Formatter Extension hasn't been updated since 2025-04-03, raising concerns about compatibility with future Chrome versions.

Active development matters for security patches and Chrome API changes. Extensions that fall behind often break unexpectedly, disrupting your development environment when you need them most.

Memory Efficiency vs Features

JSON Formatter Extension wins on resource usage with its 61.29KiB footprint compared to JSON Formatter Pro's 738KiB size. This 12x difference impacts browsers with limited RAM or when running multiple developer extensions simultaneously.

JSON Formatter Pro justifies its larger size with advanced features like JSON schema validation, minification controls, and export options. The memory trade-off becomes worthwhile when these features save development time daily.

Customization Depth

JSON Formatter Pro offers 15 syntax highlighting themes and configurable indentation settings. JSON Formatter Extension provides 3 basic color schemes with limited customization options. For developers who spend hours reading JSON data, proper syntax highlighting reduces eye strain and improves error spotting.

Theme variety becomes especially valuable when working in different lighting conditions or matching your editor's color scheme. Consistent visual styling across tools improves focus and reduces context switching friction.

"JSON formatter extensions that offer multiple themes and customization options are consistently rated higher by developers who work with API data daily, as visual parsing speed directly affects debugging efficiency.". Top 5 JSON Viewer Chrome Extensions You Need To Check Out, ful.io

When to Choose Each

Choose JSON Formatter Pro if:

You regularly work with API responses larger than 10MB
Your workflow involves complex JSON validation and schema checking
You need consistent updates and active support for Chrome compatibility
Advanced syntax highlighting and theme customization improve your productivity

Choose JSON Formatter Extension if:

You primarily format small JSON snippets under 5MB
Browser memory usage is a critical constraint in your setup
Basic formatting without advanced features meets your needs
You prefer minimal extensions that focus on single tasks

The choice often depends on your JSON complexity. Simple configuration files and small API responses work fine with either option. Large data exports and complex nested structures benefit from JSON Formatter Pro's solid processing capabilities.

For teams using Chrome extension management policies, JSON Formatter Extension's smaller size might align better with IT restrictions on extension resources.

When JSON Formatter Pro Isn't Enough

JSON Formatter Pro struggles with extremely nested objects beyond 50 levels deep, causing formatting delays even on powerful machines. Complex JSON with mixed data types and irregular structures sometimes produces formatting inconsistencies.

The extension also lacks integration with external JSON schema repositories, requiring manual validation against remote schemas. For enterprise environments with strict JSON standards, this limitation forces additional validation steps outside the browser.

Advanced users working with JSON-LD for SEO optimization might need specialized tools that understand semantic markup beyond basic JSON formatting.

Frequently Asked Questions

What is the difference between JSON Formatter Pro and JSON Formatter by Callum Locke?
JSON Formatter by Callum Locke is a popular open-source extension with a minimal footprint (61.29KiB), while JSON Formatter Pro is a more feature-rich option with 15 themes, schema validation, and 50MB+ file support. Callum Locke's extension is the right pick for developers who want something lightweight and trusted by a large community.

Which JSON formatter is more lightweight?
JSON Formatter Extension (Callum Locke) is significantly lighter at 61.29KiB versus JSON Formatter Pro's 738KiB. If your machine is RAM-constrained or you run many developer extensions simultaneously, the smaller option reduces overhead.

Which extension loads faster for large JSON files?
JSON Formatter Pro handles files up to 50MB+ without timing out, while JSON Formatter Extension has a practical limit around 5MB before performance degrades. For large API payloads or database exports, JSON Formatter Pro is the more reliable choice.

Which JSON formatter extension has more Chrome Web Store users?
JSON Formatter by Callum Locke has a substantially larger user base and more public review history. JSON Formatter Pro has a 4.8/5 rating but fewer total reviews, suggesting it's newer and growing in adoption.

The Verdict

JSON Formatter Pro delivers better performance, active development, and advanced features that justify choosing it over JSON Formatter Extension. The 4.8/5 rating and recent updates demonstrate reliability that matters for professional development work.

The larger file size becomes irrelevant when formatting speed and feature depth save hours of manual JSON manipulation. For serious development work, choose the tool that scales with your needs rather than limiting your capabilities. Try JSON Formatter Pro Free

Built by Michael Lip. More tips at zovo.one

From Raw Data to Published Reports: A Practical Guide to the Power BI Workflow

Velma Ketra Lukaya — Fri, 02 Oct 2026 13:30:42 +0000

Power BI is often described as a visualisation tool, but its real value lies in the workflow that precedes the visuals: connecting to data, assessing and cleaning it, calculating with DAX, structuring it into a well-designed model, and finally publishing and securing the result.

This article follows that workflow in the order in which it is typically learned and applied and recommends model design for a typical business intelligence project.

1. The Power BI Ecosystem

Component	Description	Role in the workflow
Power BI Desktop	Free Windows application for connecting to data (Excel, CSV and others), cleaning it, modelling it, writing DAX, and building visualisations, dashboards and reports	Authoring
Power BI Service	Cloud-based (browser) service for publishing, sharing and collaboration, with real-time updates; the full feature set is a paid licence	Publishing and sharing
Power BI Mobile	On-the-go access to reports and interaction with visuals; not used for report creation	Consumption
Power BI Report Server	Hosts reports on an organisation's own server, typically for data-security reasons	On-premises hosting

2. Introduction: Power BI as a Workflow

Power BI is a Microsoft tool that turns raw data into interactive insight. Compared with Excel, it is designed for

Larger datasets
Supports dashboards, reports and analytical calculations
Work with real-time data. It shares many function names with Excel, but its calculation language is DAX (Data Analysis Expressions).

Treating Power BI purely as a charting tool misses most of its value. Reliable reports depend on a connected sequence of stages:

data  →  preparation  →  modelling  →  analysis  →  visualisation  →  sharing

Each stage depends on the one before it. A data-quality problem that is not corrected in preparation becomes a modelling error, then a calculation error, and finally an incorrect figure on a dashboard. The sections that follow treat the stages in order.

Power BI Desktop, the environment in which data is connected, prepared, modelled and visualized.

Desktop interface

Home: Get data and Transform data.
Modeling: Manage relationships.
Insert: add a visual.
View: control layout.
Fields pane: lists all fields in the model (columns are fields; records are rows).
Visualizations pane: used to select and configure a chart or table.
Three views: Report (create and view reports), Model (manage relationships), Table (data in tabular form).

3. Data Sources

Power BI connects to a range of external sources, including Excel, CSV, SQL databases and the web. In the course, a Kenya crops dataset served as the working example. It passes through Power Query for cleaning and transformation, then into the data model, where relationships are created between tables holding different information.

 Excel ─┐
 SQL   ─┤
 CSV   ─┼──►  Power BI Desktop  ──►  Power Query  ──►  Data model
 Web   ─┘                           (clean and        (relationships)
                                     transform)

The choice of source influences how much preparation is required, which is why inspection (Section 4) follows immediately after import.

4. Importing and Inspecting the Raw Dataset

Importing data is the beginning of the work. Before any analysis, the dataset should be examined in Power Query, which has four main areas:

Ribbon: transformation commands.
Query pane: lists every table; each can be worked on individually.
Data preview: the rows and columns of the selected table.
Applied Steps: a recorded list of every change, which also functions as an undo mechanism.

Common data-quality problems

Numbers stored as text. A numeric-looking column that is left-aligned in the preview is usually being treated as text.
Unrecognised dates.
Missing values and pseudo-blanks: N/A, Error, blank, null.
Numeric columns with the wrong data type.
Duplicate records.

A missing value is not automatically zero

The meaning of a missing value depends on the column. A blank in a pest-control column may be a genuine answer (no pest control used), whereas a blank in a county column is simply absent information. Replacing every blank with 0 converts "unknown" into "zero" and distorts averages and totals later in the analysis.

Identifiers are text, not numbers

Some values that look numeric should be stored as text: ID numbers, employee IDs, phone numbers and coordinates. They are never summed or averaged, and numeric storage can strip leading zeros. Data types should reflect what a column means.

A practical convention for blanks, by column type:

Numeric column: leave as null, or replace with a number.
Text column: use a consistent label such as N/A or Not provided.

5. Cleaning and Transforming Data in Power Query

5.1 Data types and errors

When a column intended to be numeric or date-typed contains text, errors appear because of the mismatch. An error can be handled in several ways:

replace with null/blank;
replace with the actual value, if known;
remove the row;
replace with the median or mean (for example, calculated without outliers).

To correct a data type, select the column and use Transform → Detect data type.

5.2 Consistent categories

Use one name to represent a county.
Use one category to represent blank, unknown and similar values.
Fill missing values with the mode only when filling is unavoidable.

5.3 Duplicates

Duplicates are removed using a unique ID: rows sharing the same unique ID represent the same record.

5.4 Formatting and applying changes

Use Transform columns → Format for text clean-up, then Close & Apply to load the result into the model. Every action is recorded in Applied Steps, so the process is reversible and repeatable.

Why this precedes modelling

Everything downstream trusts this stage. Relationships fail on keys with inconsistent spellings, averages are wrong if blanks were silently converted to zeros, and counts are inflated by duplicates.

6. DAX: From Preparation to Analysis

DAX (Data Analysis Expressions) is the formula language Power BI uses to create calculations: calculated columns, measures and calculated tables. Where Excel formulas operate on cells, DAX operates on columns and tables and responds to filters (the filter context), a concept that recurs throughout the following sections.
DAX functions fall into six categories:

Category	Purpose
Aggregate	Mathematical and statistical summaries
Filter	Modify or create filter context and tables
Logical	Evaluate conditions to support decisions and categorisation
Text	Format and clean text
Date and Time	Extract and compare date components
Time Intelligence	Period-based calculations (see note in Section 7.9)

7. DAX Function Categories

7.1 Aggregate functions

Function	Behaviour
`SUM`	Adds all numbers in a column
`AVERAGE`	Arithmetic mean
`MIN`, `MAX`	Lowest and highest value
`MEDIAN`	Middle value of the data
`COUNT`, `COUNTA`	Count non-blank values
`COUNTROWS`	Counts rows in a table
`DISTINCTCOUNT`	Counts distinct values
`COUNTBLANK`	Counts blanks
Standard deviation, variance	Measures of spread

Mean versus median. The mean is affected by outliers; the median is not. When the mean and median are close, the data is approximately normally distributed. For skewed variables (left- or right-skewed), the median often represents a typical value better.

Iterators: SUMX and AVERAGEX. These evaluate an expression for each row of a table and then sum or average the results. They are used when a quantity is not stored in the table but can be derived from columns that are, for example revenue computed from yield and price. The table reference comes first, followed by the expression. The calculation requires knowing what constitutes the value, not the value itself, and blanks in the inputs will change the result.

Illustrative example

Total Revenue = SUMX( Crops, Crops[Yield] * Crops[Market Price] )

Arithmetic versus geometric mean. AVERAGE is the ordinary arithmetic mean. GEOMEAN multiplies the values and takes the nth root, and is appropriate for ratios, growth rates and compounding changes.

7.2 Arithmetic functions

Arithmetic in DAX is mostly expressed through operators and expressions such as the SUMX example above. DIVIDE is the safe division function and handles division by zero gracefully, which is useful in ratio measures.

7.3 Filter functions

Filter functions ensure that calculations reflect the intended filter context.

FILTER returns a subset of a table that meets a condition. The result is a new table that can be named and used like any other table.
ALL ignores the active filters.
ALLEXCEPT removes all filters except those on a specified column.
A check for whether a column is filtered to a single value (HASONEVALUE) returns TRUE when the column has exactly one value in the current context.
CALCULATE is covered in Section 7.4.

Syntax conventions

FILTER( Table, Column = "value" )          -- e.g. only Kericho county

Illustrative example:

Kericho Only =
FILTER( 'Kenya_Crops_Dataset 5', 'Kenya_Crops_Dataset 5'[County] = "Kericho" )

&& is logical AND (all conditions must be met) and must be written with two ampersands; a single & is concatenation.

|| is logical OR that requires only one of the stated conditions to be met to return results of the DAX function

For several values from one column, use the IN operator with curly brackets, for example {"Nairobi", "Meru", "Nakuru"}.

Excel's conditional aggregations (SUMIF, AVERAGEIF) have no direct DAX functions of the same name; the equivalent is built with CALCULATE.

7.4 CALCULATE

CALCULATE evaluates an expression in a modified filter context. The expression is typically an aggregate (sum, average, min, max). Because it returns a single value, it is used to create a measure. It accepts multiple filter conditions separated by commas; for two conditions on the same column, use IN.

Examples of questions answered with CALCULATE:

total yield for a given county;
market price for Nairobi;
total cost of production for farmers who planted more than 10 acres of maize in Kiambu county;
how many farmers made a loss;
average market price of tomatoes planted during the dry season.

Sales Today = CALCULATE( SUM( Sales[Sales] ), Sales[Date] = TODAY() )

If a total-sales measure already exists, it can be reused:

Sales Today = CALCULATE( [Total Sales], Sales[Date] = TODAY() )

Illustrative example:

Kiambu Maize Cost =
CALCULATE(
    SUM( Crops[Cost of Production] ),
    Crops[County] = "Kiambu",
    Crops[Crop Type] = "Maize",
    Crops[Planted Area] > 10
)

7.5 ALL and removing filters

To prevent a KPI from being affected by slicers or filters, apply ALL to the relevant table or columns. ALLEXCEPT ignores filters on every column except the one named (for example, county).
e.g a measure named Total Revenue all county, which sums Revenue (KES) while applying ALL to the County and Crop Type columns, so the card ignores slicers on those columns.

Total Revenue all county =
CALCULATE(
    SUM( 'Kenya_Crops_Dataset 5'[Revenue (KES)] ),
    ALL( 'Kenya_Crops_Dataset 5'[County] ),
    ALL( 'Kenya_Crops_Dataset 5'[Crop Type] )
)

A measure using ALLEXCEPT so that the card ignores all filters except County and Crop Type filters.

7.6 Logical functions

Logical functions evaluate conditions and return values depending on whether those conditions are true or false. They support decision-making, filtering and categorising data (for example profitable or non-profitable; low, no or high profit).

Function	Behaviour
`IF`	Tests a condition; two conditions, two outputs
Nested `IF`	More than two conditions and outputs
`AND`	True only if both conditions are true
`OR`	True if at least one condition is true
`NOT`	Reverses the logical value
`SWITCH`	A cleaner alternative to nested `IF`; evaluates an expression against matching values; `SWITCH( TRUE(), … )` allows multiple conditions
`ISBLANK`	Tests for blanks, usually inside `IF`

Two points matter in practice. Blanks fall into the "else" branch of an IF, so ISBLANK should be used where blanks need separate handling. And AND/OR used alone return only TRUE/FALSE; nested in an IF, the output can be any quoted text.

Illustrative example:

Profit Category =
IF( ISBLANK( Crops[Profit] ), "Not recorded",
    IF( Crops[Profit] < 0, "Loss", "Profitable" ) )

Use of AND

Use of OR

7.7 Text functions

Text functions clean and format text: LOWER, UPPER, PROPER, CONCATENATE/CONCAT, SPLIT, TRIM, CLEAN, LEFT and RIGHT. Because they are data-preparation tasks, they are generally better applied in Power Query.

TRIM removes extra leading or trailing spaces.
CONCATENATE joins two text values without a delimiter; for more values, or to insert a space or word, use the & operator.

Illustrative example:

Full Label = Crops[County] & " - " & Crops[Crop Type]

Without space

With space

7.8 Date and time functions

Need	Function
Current date / date and time	`TODAY`, `NOW`
Extract components (numeric)	`YEAR`, `MONTH`, `DAY`, `HOUR`, `MINUTE`, `SECOND`
Text form of a component	`FORMAT( date, "YYYY" )`, `"YY"`; `"MMMM"` full month name, `"MMM"` short; `"DDDD"` day name
Quarter	`FORMAT( date, "Q" )` (see appendix)
Difference between two dates	`DATEDIFF( date1, date2, interval )` with interval day, month or year
Working days between dates	`NETWORKDAYS` (excludes weekends, optionally holidays)
Build a date	`DATE`
Week number	`WEEKNUM`; return type 1 = week starts Sunday, 2 = week starts Monday

Illustrative example:

Order Month Name = FORMAT( Orders[Order Date], "MMMM" )
Days to Delivery = DATEDIFF( Orders[Order Date], Orders[Delivery Date], DAY )

7.9 Time intelligence

Time-intelligence functions form a category in the course structure, but no specific functions or examples were covered, so none are presented here as demonstrated content.

8. Understanding DAX Outputs

Before writing a formula, the first question is: what output is required?

Required output	DAX object	Notes
A single value	Measure	Appears in the Fields pane; usually shown in a Card visual
A column in which each row is evaluated separately	Calculated column	Adds an entire column to a table
A new table	Calculated table	For example, data for a specific date or specific counties

A measure is evaluated according to the filter context in which it is used, so it responds to slicers and to the rows and columns of a visual. A calculated column is evaluated row by row when the model is refreshed. Choosing the wrong object type is a common source of errors even when the formula itself is correct.

9. From DAX Functions to Business Questions

DAX is not an exercise in memorising formulas. The essential step is understanding the question and identifying what to filter, what to aggregate and what output is needed.

Business question	Approach
Total sales for today	Filter the sum to that date: `CALCULATE( SUM( sales ), date = TODAY() )`, or `CALCULATE( [Total Sales], date = TODAY() )`
How many sales	Count rows or transactions
How many customers	Counts people who completed transactions, regardless of how many items each bought; `COUNTROWS` or `COUNT` on the customer ID. Different approaches can give the same answer
*How many different* customers**	A distinct count (`DISTINCTCOUNT`)
Total yield or market price for a county	`CALCULATE` with a county filter
Average market price for tomatoes in the dry season	`CALCULATE` with `AVERAGE` and two conditions

These calculations assume tables that are organised sensibly. If the same customer is recorded under several spellings, "how many different customers" cannot be answered correctly. This is the bridge to data modelling.

10. Data Modelling: Structuring Data for Analysis

Data modelling is the process of organizing tables and defining the relationships between them so that data can be correctly filtered, analyzed and summarized. It involves two tasks:

defining and organizing tables;
defining relationships between tables.

Why it matters

Better reporting structure.
Reduced repetition.
Fewer errors.
Ease of creating reports.

A well-designed model also keeps DAX simple, supports query performance (each visual issues a query against the model, and fact/dimension designs suit how those queries filter, group and summarise), improves readability and maintainability, and scales more gracefully as new subject areas are added.

11. Flat Table

A flat table holds all information in one table. The Kenya crops dataset began this way: county, crop type, planted area, revenue and price all sit in each row.

FLAT TABLE   (illustrative diagram)
┌────────────────────────────────────────────────────────────┐
│ County | Crop  | Farmer | Season | Area | Yield | Revenue … │
│ Kiambu | Maize | F001   | Dry    | 12   | …     | …         │
│ Kiambu | Maize | F002   | Dry    | 8    | …     | …         │
│ Kericho| Tea   | F003   | Wet    | 20   | …     | …         │
└────────────────────────────────────────────────────────────┘
   descriptive text is repeated on every row

Aspect	Assessment
Advantages	Simple to understand; quick to start; no relationships to configure; adequate for a small, single-topic dataset
Disadvantages	Descriptive values repeat on every row; corrections must be made in many places; tables become very wide; descriptions and measurements are mixed
When appropriate	Small, one-off analyses with a single subject
Power BI implications	More repeated text is stored; no reusable dimensions to filter from; scales poorly as subjects are added

The multi-table alternative stores facts (such as transactions or hospital visits) separately from detail tables (such as doctor details), linked to the main table to avoid repetition.

12. Fact Tables and Dimension Tables

Fact table

A fact table:

Keeps the specific records, events or transactions to be analyzed (for example, all patients who visit a hospital);
Measures something;
Contains numerical records, metrics and quantities;
Allows the grain of the data to be understood;
Contains both primary and foreign keys.

Grain (granularity) is a description of what each record in a table represents. It indicates what was done. The test for any fact table is: what does one row represent? In the hospital example, one row of FactVisit represents one patient visit. If the grain is unclear, counts and totals become unreliable; for instance, if a visit were repeated once per test performed, "number of visits" would be overstated.

Generic examples : FactSales (one row per sale line), FactOrders (one row per order), FactTransactions (one row per payment).

Dimension table

A dimension table

Describes something, giving context for the events held in the fact table (for example, patient details);
Contains descriptive text content;
Has a primary key.

Example: DimPatient, DimDoctor, DimDepartment, DimCustomer, DimProduct, DimDate, DimLocation.

	Fact table	Dimension table
Answers	What happened? How much?	Who, what, where, when?
Contents	Numbers, metrics, keys	Descriptive attributes
One row represents	One event (the grain)	One entity
Keys	Primary and foreign keys	Primary key
Typical size	Many rows	Fewer rows

Separating them means each description is stored once, while the fact table stays focused on events.

13. Star Schema

A star schema consists of one central fact table surrounded by dimension tables, each connected to the fact table.

                 DimDepartment
                       │
                       │
   (other dimension) ── FactVisit ── DimPatient
                       │
                       │
                   DimDoctor

Advantages

Simple, readable model diagram.
Descriptions are stored once, reducing repetition.
Dimensions filter and the fact table is summarised, which matches the way Power BI visuals query a model.
Simpler DAX: measures aggregate fact columns while slicers come from dimensions.
Easy report creation.

Disadvantages

Requires planning, since a flat table must be split into facts and dimensions.
Some descriptive values still repeat within a dimension (for example, a county name for every patient in that county).

Appropriate use: most business intelligence and reporting models.

14. Snowflake Schema

A snowflake schema works like a star schema, but the dimension tables are broken down into several smaller related tables. The structure is hierarchical: fact → dimension → sub-dimension.

 DimCountry ── DimCounty ── DimPatient ── FactVisit

	Star	Snowflake
Dimensions	One table per dimension	Dimension split into related sub-tables
Redundancy	Some repetition inside dimensions	Less repetition (more normalized)
Relationship hops from slicer to fact	One	Two or more
Model diagram	Compact	Larger, more tables

Advantages:
-Less repeated data within dimensions;
-Explicit hierarchies;
-A shared lookup (such as county) can be maintained in one place.

Disadvantages:
-More tables and relationships,
-Longer filter paths, and a
-Busier model that is harder for report builders to navigate.

Appropriate situations: when a sub-dimension is shared by several dimensions, or when the source data is already normalized.

15. Comparing Flat, Star and Snowflake Schemas

Feature	Flat table	Star schema	Snowflake schema
Structure	Everything in one table	One fact table plus dimension tables	Fact table plus dimensions split into sub-tables
Number of tables	1	Few	Most
Redundancy	High	Low to moderate	Lowest
Model complexity	Simplest to set up; harder to maintain as it grows	Moderate; easy to read	Highest
DAX / reporting	Adequate for simple cases; filtering limited to that one table	Simple: measures on facts, slicers from dimensions	Works, but filters pass through more relationships
Performance	Wide tables with repeated text can be inefficient	Recommended by Microsoft for Power BI	Extra relationship hops; usually no benefit unless the structure is needed
Scalability	Poor	Good	Good, with growing complexity
Maintainability	Corrections repeated across many rows	Fix a description once	Good for hierarchies; heavier to manage
Appropriate use	Small, single-topic analysis	Most BI projects	Shared hierarchies; normalised sources

16. Primary Keys and Foreign Keys

A primary key is a unique identifier column that gives each record in a table a unique identity.
A foreign key is a primary key from one table that is used in a different table.
A common column is required before a relationship can be created.

Example. CustomerID is unique in DimCustomer (one row per customer) but may appear many times in FactSales, because one customer can make many purchases. DimCustomer[CustomerID] is therefore a primary key, FactSales[CustomerID] is a foreign key, and the relationship is one-to-many.

Illustrative example:

DimCustomer

CustomerID (PK)	CustomerName
C1	Amina
C2	Brian

FactSales

SaleID	CustomerID (FK)	Amount
S1	C1	500
S2	C1	300
S3	C2	450

Referential integrity means every foreign-key value in the fact table has a matching primary key in the dimension. If FactSales contained C9 but DimCustomer had no C9, those sales would not attach to any customer and would appear under a blank member in visuals.

17. Relationships in Power BI

A relationship shows how tables are connected. It is created by matching a primary key in a dimension table with a foreign key in the fact table. Relationships are necessary because, once data is distributed across tables, Power BI needs to know which fact rows belong to which dimension rows. They are what allow filters to travel from one table to another.

Key properties:

Keys: primary key ↔ foreign key.
Cardinality: how rows in one table relate to rows in another (Section 18).
Cross-filter direction: single or both (Section 19).
Referential integrity (Section 16).
Active and inactive relationships: only one relationship between two tables can be active at a time. Additional relationships can exist but are inactive (shown as dashed lines in Model view) and do not filter unless a DAX measure activates them with USERELATIONSHIP. A relationship must exist and be active for columns from different tables to be used together.

18. Relationship Cardinality

Cardinality describes how data in one table relates to data in another, based on the count of matching rows. It should be distinguished from column cardinality, which refers to how unique the values in a single column are.

One-to-many (1:*), the most common type

A single row in one table matches multiple rows in the other.

Example: one patient (dimension) → many visits (fact); illustrative: one DimCustomer row → many FactSales rows.
Common in dimensional models because every dimension-to-fact link is of this type: the "one" side holds unique keys and the "many" side holds the events.

DimCustomer (1) ─────────────< FactSales (*)

Many-to-one is the same relationship read from the other direction.

One-to-one (1:1)

One row in a table links to only one row in another table. It arises when there is no separate information that justifies splitting the table, and it relates to normalisation.

Illustrative example: Employee and EmployeeBadge, with exactly one badge record per employee.
Before using it, consider whether the two tables should simply be combined into one (for example by a merge in Power Query). Filters flow in both directions in a 1:1 relationship.

Employee (1) ─────────── (1) EmployeeBadge

Many-to-many (:)

Multiple rows in one table link to multiple rows in another.

Illustrative example: students and courses.
It requires careful modelling because neither side has unique keys, so filters can match many rows on both sides and totals can become hard to interpret. A bridge (junction) table converts it into two one-to-many relationships.

Students (*) ───────────── (*) Courses
   safer:  Students (1)──<  Enrolments  >──(1) Courses

19. Filter Direction

Filter direction controls which table filters which when a user interacts with a slicer or visual.

Single-direction filtering

Filters flow from the "one" side (dimension) to the "many" side (fact).

DimProduct
    │
    ▼
FactSales

Selecting a product in a DimProduct slicer filters the corresponding rows in FactSales, so a revenue card then shows only that product's revenue (illustrative). This is the default for a one-to-many relationship and the preferred setting in a star schema.

Bidirectional filtering

DimProduct
    ▲
    ▼
FactSales

Filters flow in both directions. It should be used sparingly because it can cause:

ambiguous filter paths, where multiple routes between two tables leave it unclear which one Power BI should use, potentially deactivating a relationship or producing route-dependent results;
unnecessary complexity, since each bidirectional relationship makes the model harder to reason about;
unexpected filtering behaviour, as slicers begin filtering one another and numbers change in ways report users do not expect;
reduced performance on large models.

Guideline: keep relationships single-direction from dimension to fact, and use Both only for a specific, understood reason.

20. Establishing Relationships in Power BI

Identify the matching keys: the primary key in the dimension and the foreign key in the fact table.
Create the relationship by either:
- drag and drop, from primary key to foreign key in Model view; or
- Modeling → Manage relationships on the ribbon.
Set the cardinality and the cross-filter direction.

Preparing keys when none exist

When a flat table is split into dimensions, the following workflow applies:

Clean the data.
Create dimension tables, working out which columns belong to each dimension.
Give each dimension a unique identifier. If none exists, create one: right-click on the table → Add index column (from 1).

In the original table, find and replace the descriptive name with the unique index, which then serves as the foreign key.
Clean the data again and check for errors.
Create relationships and ensure they are active.
Calculate (DAX).
Display (dashboard).

The same result can also be achieved by merging the original table with the new dimension in Power Query, which leads to the next topic.

21. Joins in Power Query

Relationships connect tables inside the model. A join combines tables earlier, during data preparation. A join combines two tables using matching columns identified through keys, and is performed in Power Query because the table is being transformed to create a new, merged table.
In Power Query the command is Merge Queries. Which table's rows are kept in full depends on the join type.

Example used for all six joins

Illustrative example
Join key: CustomerID.

Customers (left table)

CustomerID	CustomerName
C1	Amina
C2	Brian
C3	Chebet

Orders (right table)

OrderID	CustomerID
O1	C1
O2	C1
O3	C4

Brian (C2) and Chebet (C3) have no orders, and order O3 belongs to C4, who is not in the Customers table.

21.1 Inner join

Keeps only matching records in both tables. Use when only records existing on both sides are wanted.

CustomerID	CustomerName	OrderID
C1	Amina	O1
C1	Amina	O2

Customers ∩ Orders         only the overlap

21.2 Left outer join

Keeps all rows from the left (first) table and only the matching rows from the right. Use to list all customers whether or not they ordered.

CustomerID	CustomerName	OrderID
C1	Amina	O1
C1	Amina	O2
C2	Brian	null
C3	Chebet	null

all of Customers + matching Orders

21.3 Right outer join

Keeps all rows from the right (second) table and the matching rows from the left.

CustomerID	CustomerName	OrderID
C1	Amina	O1
C1	Amina	O2
C4	null	O3

matching Customers + all of Orders

21.4 Full outer join

Keeps all records from both tables, whether or not they match.

CustomerID	CustomerName	OrderID
C1	Amina	O1
C1	Amina	O2
C2	Brian	null
C3	Chebet	null
C4	null	O3

Customers ∪ Orders         everything

21.5 Left anti join

Keeps only the rows in the left table that have no match in the right. Useful for finding customers who never ordered, and for data-quality checks.

CustomerID	CustomerName
C2	Brian
C3	Chebet

Customers − Orders         left only

21.6 Right anti join

Keeps only the rows in the right table that have no match in the left. Useful for finding orders whose customer is missing, which is a referential-integrity problem (Section 16).

OrderID	CustomerID
O3	C4

Orders − Customers         right only

Summary

Join	Rows kept	Result in the example
Inner	Matches only	2 rows
Left outer	All left + matching right	4 rows
Right outer	All right + matching left	3 rows
Full outer	Everything	5 rows
Left anti	Left rows with no match	2 rows
Right anti	Right rows with no match	1 row

22. Join Output Illustrations

 Left table = L,  Right table = R,  overlap = matched rows

 Inner       :        [ L ∩ R ]
 Left outer  :   [ L ........ ∩ ]       all of L, matched R
 Right outer :        [ ∩ ........ R ]   matched L, all of R
 Full outer  :   [ L ........ ∩ ........ R ]
 Left anti   :   [ L only ]
 Right anti  :                [ R only ]

23. Power Query Joins vs Power BI Relationships

Both mechanisms connect tables through matching keys, but they occur at different stages and do different things.

 Power Query                         Data model
 ───────────                         ──────────
 Merge / Join  ──►  Data           ──►  Relationships  ──►  DAX / Visuals
 (combine rows)     transformation       (connect tables)
 "preparation"                           "modelling"

Does a Power Query merge physically combine data? Yes. A merge creates a new table, or adds columns to an existing one, in which matched rows sit side by side. After Close & Apply, that wider table is what is loaded into the model.

Does a relationship physically combine tables? No. A relationship creates no new table and moves no data. Both tables remain separate. It is a rule in the model stating that when one table is filtered, the other is filtered through a shared key. Power BI applies it at calculation time, when a visual or measure needs columns from both tables.

When does each happen? Joins happen during preparation (Power Query); relationships are defined during modelling (Model view), after loading.

When is a merge preferable to a relationship?

To add a column from another table, such as bringing a county name into a table that only held a county code (illustrative).
To flatten a snowflake by merging sub-dimensions into one dimension.
To filter rows by existence in another table, using anti joins.
To bring a key into a fact table when creating dimensions.

A relationship is preferable when tables hold different kinds of information (dimension and fact) and should stay separate while still filtering one another.

What can excessive merging cause?

Wider tables with more columns.
Greater redundancy, since descriptions repeat on every row (the flat-table problem).
Loss of dimensional structure, because facts and dimensions blend into one table and clean, reusable dimensions disappear.
Higher model complexity and maintenance cost, and row multiplication if the key is not unique on one side of the merge.

Why do fact and dimension tables stay separate? A star schema works because dimensions filter and facts summarise. Merging everything into one table abandons that structure. Keeping the tables separate and relating them stores each description once, keeps the fact table focused on its grain, and keeps DAX simple.

	Power Query merge (join)	Power BI relationship
Stage	Data preparation	Data modelling
Location	Power Query Editor	Model view
Physically combines data?	Yes: creates a merged table	No: tables stay separate
Output	A new table or extra columns	A filter path between tables
Controlled by	Join kind (six types)	Cardinality and filter direction
When applied	At refresh / load	At query time, when visuals and measures run
Best for	Cleaning, enriching, flattening, checking	Linking facts to dimensions

In short: join during preparation; relate during modelling.

24. From Model to Visualisation

Once data has been cleaned, modelled, related and calculated, reports can be built. In Report view, a chart type is selected and the fields to plot are added. Visuals are interactive: selecting part of one visual filters the others, including KPIs, and even the components of a bar are interactive.

Visual	Purpose
Bar chart	Comparing categories
Pie / Donut / Treemap	Part-to-whole relationships; treemaps suit many categories
Scatter	Relationship between two numeric variables
Card	Displaying a KPI or single headline value (typically a measure)
Map	Geographic data (bubble, filled and shape maps)
Gauge / KPI	Targets and comparison
Slicer	Filtering by user selection

These visuals depend on the earlier stages. The card shows a CALCULATE measure (Section 7.4); the slicer filters through a relationship (Sections 17 and 19); chart categories come from clean dimension columns (Section 5); and the Total Revenue all county card (Section 7.5) shows how ALL makes a KPI ignore slicers deliberately.

25. Publishing to Power BI Service

Key terms

.PBIX: the Power BI Desktop file.
Semantic model: the entire data ecosystem: tables, relationships and connections, measures, and the model itself.
Report: the visuals, graphs and dashboards that report users read to evaluate the data.
Workspace: a collaborative area in Power BI Service where a team creates, manages and stores Power BI content.
Power BI Service: the browser version of Power BI, used to create, share and consume content.

What publishing does

Publishing sends the report and the semantic model from the .pbix file to the selected workspace.

Power BI Desktop (.pbix)
          │
          ▼  Home → Publish → select destination
Power BI Service
          │
          ▼
      Workspace
          │
          ▼
Semantic model  +  Report
          │
          ▼
Sharing / Collaboration  (workspace roles and security)

The semantic model is the same model built in Model view, now hosted in the cloud, so earlier modelling decisions affect every report built on it.

Steps

Before publishing: sign in to Power BI from the Desktop app, then sign in to Power BI Service online.
Create a workspace: open the left pane → Workspaces and complete the details (name, description, image, domain, Power BI Pro where external users need access, contact list).
Publish: in Desktop, Home → Publish → select destination.
Open the workspace and refresh to see the report and semantic model.
Share: subscriptions allow a particular report page to be shared with an intended person; a report with multiple pages can also be shared as a link.

26. Workspace Roles and Security

Workspace roles

Role	Description
Admin	Controls the workspace and access
Member	Can collaborate and publish into the workspace
Contributor	Can create and change content, with limited access functions
Viewer	Can consume content without editing

Exact permission lists should be confirmed against current Microsoft documentation.

Row-level security (RLS)

Row-level security controls which rows of data a user can see based on their assigned role, to keep data secure. Rules specify who needs access to what, and roles can be created before the report is shared and tested with View as.

Define roles and their DAX filter rules in Power BI Desktop (Modeling → Manage roles).
Publish, then assign users or groups to each role in Power BI Service. A defined role is not enforced until users are assigned.
RLS applies to Viewers. Members of the Admin, Member and Contributor roles have edit permission on the semantic model, so RLS does not restrict them.
Test with View as before sharing.

RLS filters travel across relationships like any other filter and by default follow single direction, which is a further reason to keep filter directions simple.

27. Recommended Power BI Model

For a typical business intelligence project, a star schema is the recommended design. The reasoning is that it fits how Power BI queries models and the types of questions BI projects ask, not that it is universally superior.

Criterion	Flat table	Star schema	Snowflake
Query / report performance	Repeated text bloats wide tables	Matches how Power BI visuals query the model; recommended by Microsoft	Extra hops between tables
DAX simplicity	Simple at first; awkward as questions grow	Simple: measures on facts, slicers from dimensions	More relationships to reason about
Readability	Hard to tell what each column represents	Clear: facts in the centre, dimensions around	Larger diagram
Scalability	Poor	Good: add a dimension or fact	Good, with added complexity
Data redundancy	High	Low	Lowest
Maintainability	Same value fixed in many rows	Fix a description once	Good for hierarchies; heavier to manage
Ease of report creation	One place for everything, but limited filtering	Easy: dimension fields for slicers and axes, fact fields for values	Builders must understand the chain
Filter propagation	Nothing to manage	Predictable: dimension → fact	Passes through several tables
Model complexity	Lowest initially	Moderate	Highest

When another design is appropriate

Flat table: a small, one-off analysis with one subject, where building dimensions costs more than it returns.
Snowflake: when a sub-dimension is genuinely shared (for example, a county table used by both patients and clinics) or the source is already normalised and flattening is not worthwhile. Merging the sub-tables in Power Query to obtain a simpler star should be considered first.

Recommended relationship design

One fact table at the centre, with a grain that can be stated in one sentence (for example, one row per visit).
One dimension table per descriptive subject (patient, doctor, department, date, location), each with a unique primary key.
Foreign keys in the fact table matching those primary keys. Where no natural key exists, create an index column and carry it into the fact table.
Cardinality: one-to-many (1:*) from each dimension to the fact table. Avoid many-to-many unless unavoidable, using a bridge table where needed, and consider merging one-to-one pairs into a single table.
Filter direction: single, from dimension to fact. Use Both only for a specific, understood reason, to avoid ambiguous paths.
Active relationships on the main path; an inactive relationship only where a measure deliberately uses USERELATIONSHIP.
Referential integrity checked in Power Query using a right anti join (fact keys with no dimension match) before loading.

 DimDepartment   DimDoctor
        \          /
         \        /
 DimPatient ── FactVisit ── DimDate
   (1)──────────►(*)         (1)──►(*)      single-direction, 1:* from every dimension

(Illustrative design based on the hospital example, with a date dimension added.)

28. The Complete Workflow

 1. Understand the problem / question
              ↓
 2. Identify the data source
              ↓
 3. Import / connect data
              ↓
 4. Inspect the raw data
              ↓
 5. Clean and transform with Power Query
              ↓
 6. Understand the structure of the data
              ↓
 7. Build the data model
              ↓
 8. Create fact and dimension tables
              ↓
 9. Establish relationships
              ↓
10. Apply appropriate cardinality / filter direction
              ↓
11. Use DAX to calculate and analyse
              ↓
12. Create visualisations
              ↓
13. Build the report
              ↓
14. Publish to Power BI Service
              ↓
15. Share, secure and collaborate

How the stages connect

Steps 1–3: The business question determines which data is needed and where it lives. Import brings a copy into Desktop.
Steps 4–5: Imported data is rarely analysis-ready. Data types, blanks, errors and duplicates are resolved in Power Query, with Applied Steps recording each action. Joins are available here for combining or checking tables.
Steps 6–8: Clean data is organised: measurements into a fact table with a clear grain, descriptions into dimension tables, usually in a star schema.
Steps 9–10: Primary keys in dimensions link to foreign keys in the fact table through one-to-many, single-direction relationships. The tables then work as one model without being merged.
Step 11: DAX operates on top of the model. CALCULATE, FILTER and ALL depend on filters moving through relationships, so the better the model, the simpler the DAX.
Steps 12–13: Visuals present measures and fields and respond to the filters defined in the model.
Steps 14–15: The .pbix is published to a workspace as a semantic model and report, with roles and RLS governing who can do and see what.

29. Conclusion

Power BI is best understood as a chain of dependent practices rather than a set of separate features. Data enters through connections to sources such as Excel, CSV, SQL and the web. Its quality must be established in Power Query before it can be trusted. DAX then turns clean data into answers, but its effectiveness depends on a well-structured model: fact tables with a defined grain, dimension tables with unique keys, one-to-many relationships and carefully chosen filter directions. Joins and relationships are complementary tools applied at different stages: joins combine data while it is being prepared, and relationships connect tables once they are loaded. Reports, dashboards and publishing sit at the end of the chain, so their reliability reflects the care taken in every earlier stage.

For most business intelligence projects, a star schema built on clean, correctly typed data offers the best balance of performance, simplicity, readability and maintainability.

Extract Job Listings from Indeed — Python Tutorial 2026

RAZIX DEVIL NEMESIS (Loki) — Fri, 02 Oct 2026 13:15:48 +0000

Extract Job Listings from Indeed — Python Tutorial 2026

Indeed is the world's largest job search engine with 250M+ monthly visitors. Here's how to extract job listing data programmatically.

Why Scrape Indeed?

Market salary analysis
Job demand trends
Recruitment automation
Competitive intelligence

Quick Start with Apify

Use the Indeed Job Scraper on Apify — no coding needed. Just enter keywords and location, get structured JSON back.

Python Method

import httpx
from bs4 import BeautifulSoup

def search_indeed(keyword, location):
    url = f"https://www.indeed.com/jobs?q={keyword}&l={location}"
    resp = httpx.get(url, headers={ "User-Agent": "Mozilla/5.0" })
    soup = BeautifulSoup(resp.text, "lxml")
    jobs = []
    for card in soup.select(".job_seen_beacon"):
        title = card.select_one(".jobTitle")
        company = card.select_one(".companyName")
        if title and company:
            jobs.append({
                "title": title.get_text(strip=True),
                "company": company.get_text(strip=True),
            })
    return jobs

What Data You Can Extract

Job title and description
Company name and rating
Salary range
Location
Posting date
Application link

Best Practices

Use polite delays between requests
Rotate user agents
Consider using a proxy service
Check Indeed's terms of service

Built by an AI agent earning passively. Deploy your own: Omnincome Agent

Why Your PDF Text Extraction Returns an Empty String

Nikolas Dimitroulakis — Fri, 02 Oct 2026 13:13:36 +0000

You wrote ten lines of Python, pointed it at a PDF, and got back "". No exception, no warning, nothing.

The reason is almost always the same: that PDF has no text layer. It is a picture of a document, not a document. Text-layer libraries return nothing and report no error, which makes this one of the quieter bugs you can ship.

Here is how to detect it and route around it.

Step 1: Check for a text layer before you extract

From the shell, poppler-utils answers this in one command:

pdffonts document.pdf

Fonts listed means there is a text layer. Empty output means it is a scan.

In Python, with PyMuPDF:

import fitz  # PyMuPDF

def has_text_layer(path, min_chars=50, pages_to_check=3):
    doc = fitz.open(path)
    for page in doc[:pages_to_check]:
        if len(page.get_text().strip()) >= min_chars:
            return True
    return False

Two details worth keeping:

Check several pages, not just the first. Scanned documents often carry a generated cover page with real text in front of forty pages of images. A page-one check sends the whole file down the wrong branch.

Use a character threshold, not a truthiness check. Plenty of scans carry a few stray characters from a header or a watermark. That is enough to pass if text: and nothing more.

Step 2: Text-layer PDFs

PyMuPDF is the fastest general-purpose option:

def extract_text_layer(path):
    doc = fitz.open(path)
    return "\n".join(page.get_text() for page in doc)

If columns come out interleaved, pdftotext -layout document.pdf document.txt often does better, and pdfplumber gives you word-level coordinates when you need to reconstruct the layout yourself.

Step 3: Scanned PDFs

Scans need OCR. OCRmyPDF wraps Tesseract and writes a text layer back into the file, which means the rest of your pipeline stays unchanged:

import subprocess

def ocr_pdf(src, dst, lang="eng"):
    subprocess.run(
        ["ocrmypdf", "-l", lang, "--skip-text", src, dst],
        check=True,
    )
    return extract_text_layer(dst)

Set -l to the right language. The default is English, and accuracy drops sharply on anything else.

For documents the local stack handles badly (poor scans, forms, handwriting), a hosted OCR document data extraction API returns layout-aware parsing without a Tesseract install, and an image OCR API covers loose JPEGs and PNGs.

Step 4: Route per file

def extract(path, ocr_output="ocr.pdf"):
    if has_text_layer(path):
        return {"method": "text-layer", "text": extract_text_layer(path)}
    return {"method": "ocr", "text": ocr_pdf(path, ocr_output)}

Return the method alongside the text and log it. The day output looks wrong, you will know in one query whether the problem was your extraction or the source document.

Calling it as a hosted API

If you would rather not run any of this, the Extract Text from PDF API takes a remote URL or a file upload and returns the text in a single data field:

curl -X POST "https://api.eu.apyhub.com/extract/text/pdf-url" \
  -H "apy-token: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/report.pdf"}'

It also accepts a page range and coordinate bounds, which is useful when you only need one section of a long report.

Which method for which job

Method	Good at	Trade-off
`pdftotext -layout`	One-off text-layer extraction, keeps columns roughly intact	No structure, little control over output
PyMuPDF	Fast bulk extraction with coordinates	Tables lose row structure
pdfplumber	Word positions, lined tables	Noticeably slower on large files
ocrmypdf + Tesseract	Clean scans, free, runs locally	Accuracy falls off below 300 DPI and on skewed pages
Cloud OCR (Azure, Google, AWS)	Poor scans, forms, handwriting	Per-page cost, data leaves your infrastructure
Extract Text from PDF API	Text-layer PDFs by URL or upload, page ranges and coordinate bounds, one call, nothing to install	50 atoms per call

A note on tables

Tables are their own problem. Lined tables (visible borders) extract reasonably well with pdfplumber or camelot. Unlined tables usually do not, and OCR makes it worse by returning text in reading order and discarding which cell each value came from.

When row and column structure matters, a table extraction API that preserves layout saves more time than tuning a parser will.

The summary

Detect the text layer per file, never per batch.
Text layer goes to PyMuPDF.
No text layer goes to OCR.
Log which branch each file took.

Most of the pain in PDF extraction comes from assuming one method handles everything. Once the routing is in place, each method only has to be good at the job it is actually given.

Every API above is also available through ApyHub MCP, so an AI agent can call them directly without a hand-written wrapper. You can try any of them from its page in the ApyHub catalog or in Voiden, the open-source API client, before writing any code.

DEV Community: tutorial

Construindo um Índice Invertido em Elixir: do zero ao TF-IDF

O que é um índice invertido?

Preparando o projeto

Passo 1: Tokenização

Passo 2: Construindo o índice

documentos |> Enum.flat_map(fn {doc_id, texto} -> ... end)

Enum.uniq() e Enum.map(fn termo -> {termo, doc_id} end)

Enum.group_by/3

Map.new/2 com Enum.sort

Testando

Passo 3: Busca booleana (AND / OR)

Busca OR

Busca AND

Uso

Passo 4: Frequência de termos e posições

tokenizar_com_posicao/1

Primeiro flat_map — desmembrando

Primeiro group_by — agrupando por termo

Segundo group_by dentro do Map.new

Resultado

Passo 5: Ranking com TF-IDF

Enum.reduce(termos, %{}, fn termo, acc -> ... end)

case Map.get(indice, termo) do ... end

idf = :math.log(total_docs / map_size(docs))

Enum.reduce(docs, acc, fn {doc_id, %{tf: tf}}, a -> ... end)

Map.update(a, doc_id, tf * idf, &(&1 + tf * idf))

Ordenação

Uso

Passo 6: Testes

Onde ir a partir daqui

Conclusão

Cloudflare libera Clef y Clef-flash bajo licencia Apache 2.0

TL;DR

Qué es Clef

Qué pasó

El auge reciente de los modelos de decisión

Cómo compite Clef en precisión y latencia

Impacto y análisis

Qué sigue

Preguntas frecuentes

¿Qué son los modelos de decisión?

¿Clef reemplaza a un LLM como GPT o Claude?

¿Clef y Clef-flash son gratis?

¿Qué diferencia hay entre Clef y Clef-flash?

¿Qué es la Jev Decision Index?

¿Puedo entrenar mi propia versión de Clef?

Referencias

Swap tokens from your agent wallet: quote first, then execute

1. Get the wallet id

2. Quote before anything moves

3. Read the policy that will check the swap

4. Execute

5. Confirm what landed

What to watch

Reimplementing a Trie Reminded Me How Autocomplete Actually Works

PNG to WebP 2026: The Complete Guide to Smarter Images

How Unicode Bubble Text Generators Work (and Where They Break)

Bubble text is character substitution

Map code points instead of storing giant lookup objects

Iterate by code point, not UTF-16 code unit

Unicode style ranges are not symmetrical

Preserve characters you cannot convert

Show every result at once

Compatibility is part of the product

The result

Takeaways

How to Make Money with AI Music: Build a Release Workflow with SongUpAI

Treat the track as a deliverable

Gate 1: check the commercial-use path

Gate 2: generate from a specific brief

Gate 3: run a listening review

Gate 4: package the project so you can deliver it again

Pick one monetization route to test

Route A: release music for listeners

Route B: create a narrowly scoped service

Route C: develop a useful audio pack

Track feedback alongside costs

Make the explanation discoverable

Ship one small, reviewable project

`documentos |> Enum.flat_map(fn {doc_id, texto} -> ... end)`

`Enum.uniq()` e `Enum.map(fn termo -> {termo, doc_id} end)`

`Enum.group_by/3`

`Map.new/2` com `Enum.sort`

`tokenizar_com_posicao/1`

Primeiro `flat_map` — desmembrando

Primeiro `group_by` — agrupando por termo

Segundo `group_by` dentro do `Map.new`

`Enum.reduce(termos, %{}, fn termo, acc -> ... end)`

`case Map.get(indice, termo) do ... end`

`idf = :math.log(total_docs / map_size(docs))`

`Enum.reduce(docs, acc, fn {doc_id, %{tf: tf}}, a -> ... end)`

`Map.update(a, doc_id, tf * idf, &(&1 + tf * idf))`