DEV Community: Lucas Aguiar

MED 2.0 for devs: entities, fields and the implementation flow

Lucas Aguiar — Mon, 24 Aug 2026 19:23:08 +0000

If you work with Pix at a fintech, one thing is certain: sooner or later you'll have to deal with fraud. And the bar the Central Bank sets for that just went up. In August 2025 the MED 2.0 rules came out (starting with Resolution BCB nº 493), the new version of Pix's Special Refund Mechanism — optional to adopt from Nov 23, 2025 and with a mandatory rollout scheduled for 2026.

In this article I want to explain, in a simple and technical way, what the MED entities are, the fields that matter in each one and how the flow works end to end — with a focus on whoever is going to write the code.

🧩 First: what MED is (and why it exists)

MED (Mecanismo Especial de Devolução — Special Refund Mechanism) is a regulatory obligation from the Central Bank for every Pix participant — banks, payment institutions, fintechs, anyone who does Pix.

The idea is simple: when fraud or an operational failure happens, there must be a standardized and auditable path to return the victim's money. Without MED, each institution would solve it its own way (or not at all).

Important: MED is not for commercial disputes, buyer's remorse, or "I sent a Pix to the wrong person because I wasn't paying attention." It's for:

Scam / fraud (phishing, social engineering, coercion)
Account takeover
Fraudulent access to channels
Institution's operational error (duplicate Pix, wrong amount from a system failure)

💡 Rule of thumb: MED = proven fraud or an error by the institution. It's not a "cancel my Pix" channel.

🚀 What changes in MED 2.0

MED 1.0 had an Achilles' heel: it only blocked and refunded at the first receiving account of the fraudulent Pix. So the scammer pulled the classic move — received the money and scattered it in a cascade across many accounts in seconds. By the time the block arrived, the money had already evaporated. According to the BCB, in 2025 only 9.3% of the disputed amount was recovered on average.

MED 2.0 tackles exactly that: it now traces the money's path beyond the first account. The big highlights:

Multi-layer tracing. DICT follows the money through the transaction graph and picks the transactions most likely to be on the drain path.
Targeted blocking. The amount is blocked at each node of the path — only what matters, not the whole account (more on this below).
Refund in its own name. The receiver's institution debits the mule account and refunds the victim directly, appearing as the payer on the statement.
Digital self-service. The harmed customer triggers the process from the app.
Standardized deadlines and UX (Timing Manual and Minimum UX Requirements).

🧱 The MED entities (and how they connect)

Before diving in, the most important thing is to understand the end-to-end flow. It's what gives meaning to each entity:

The harmed customer contacts their PSP, which opens a Funds Recovery (FundsRecovery). This is the starting point.
DICT traces the transactions from the root transaction and sends Infraction Reports (InfractionReport) to the PSPs of the accounts along the money's path.
Each receiving PSP blocks the balance (only the indicated amount) and analyzes the Infraction Report.
Once the infractions are analyzed, the paying PSP triggers the start of the Refund Requests — and the money goes back to the victim, layer by layer.
Along the way, fraud markers are recorded in DICT, feeding prevention.

So there are four entities. Understand these four and you understand MED:

Entity	Base endpoint	Role in the flow
Funds Recovery	`/funds-recoveries/`	Opens the case. The harmed customer starts it; it traces and orchestrates everything (the 2.0 highlight).
Infraction Report	`/infraction-reports/`	What each PSP in the chain receives. Triggers the block and the analysis.
Refund Request	`/refunds/`	Executes the refund of the money to the victim, layer by layer.
Fraud Marker	(automatic on agreement)	Flags receivers as fraudsters in DICT, feeding prevention.

Let's go through them one by one, in the order they appear in the flow.

🕸️ Funds Recovery (`FundsRecovery`)

This is the big novelty of MED 2.0 — and where everything begins. When the customer is a fraud victim, the paying PSP opens a Funds Recovery (POST /funds-recoveries/). From there, DICT traces the money and orchestrates blocks and refunds.

The stages (this is where the "several statuses" live)

Funds Recovery isn't a single step — it moves through stages with their own statuses, and your backend needs to react to each transition. Today the flow is fully automated: DICT does the tracing, prioritization and dispatch of the Infraction Reports with no manual step from the PSP.

Status	Stage
`CREATED`	Funds Recovery created over the root transaction
`TRACKED`	Transaction tracking graph created
`AWAITING_ANALYSIS`	Infraction Reports generated; receiving PSPs analyzing (up to 7 days)
`ANALYSED`	Infraction Reports analyzed and closed; the paying PSP can trigger the refund

After ANALYSED comes the refund (requests run sequentially) and, at the end, DICT signals completion through the FUNDS_RECOVERY_COMPLETED event.

Tracking graph parameters

Parameters that bound the graph tracing (applied by DICT's algorithm):

Parameter	What it controls
`MinTransactionAmount`	Minimum amount of each transaction considered
`MaxTransactions`	Maximum number of transactions in the graph
`HopWindow`	Time window between money entering and leaving an account
`MaxHops`	Maximum graph depth (how many layers to go down)

Main endpoints

Method + route	What it does
`POST /funds-recoveries/`	Create a Funds Recovery
`GET /funds-recoveries/{id}`	Query a Funds Recovery
`POST /funds-recoveries/{id}/tracking-graph`	Create a tracking graph
`GET /funds-recoveries/{id}/tracking-graph`	Query the graph
`POST /funds-recoveries/{id}/block`	Request the transaction blocks
`GET /funds-recoveries/{id}/infraction-reports`	List the Infraction Reports of the recovery
`POST /funds-recoveries/{id}/refund`	Request a refund within the recovery
`GET /funds-recoveries/{id}/refunds`	List the refunds of the recovery
`POST /funds-recoveries/{id}/cancel`	Cancel the recovery

🔔 Don't sit there polling. DICT has an events endpoint (GET /event-notifications) with two key signals for the recovery:

FUNDS_RECOVERY_ANALYSED → the Infraction Reports were analyzed; you evaluate the result and trigger (or not) the start of the refund.

FUNDS_RECOVERY_COMPLETED → the refund phase is over; anyone still holding blocked funds can release them (if there's no other basis to keep the block).

📢 Infraction Report (`InfractionReport`) — the block

If Funds Recovery is the customer pulling the trigger, the Infraction Report is what lands on the PSPs of the accounts along the money's path. Each prioritized transaction generates an Infraction Report for the PSP of that node, which then blocks the balance and analyzes the case.

Main fields

Field	What it is
`TransactionId`	EndToEndId of the suspicious transaction (e.g., `E999990101234...`)
`InfractionType`	Type: `FRAUD`, `REFUND_REQUEST`, `REFUND_CANCELLED`
`FundsRecoveryId`	(MED 2.0) links the report to the Funds Recovery
`InfractionAmount`	(MED 2.0) the amount that must be blocked from that transaction
`InfractionData.DebitedParticipant` / `CreditedParticipant`	ISPBs of the payer and the receiver
`InfractionData.ReportedBy`	`DEBITED_PARTICIPANT` or `CREDITED_PARTICIPANT`
`InfractionData.TransactionDate`	Transaction date-time
`InfractionData.InfractingAccountData`	Branch + account involved at the node
`TransactionDepth`	(latest) position of the transaction in the money-movement chain — how far it is from the initial fraud (DICT Manual v8.5)
`Status`	`OPEN`, `ACKNOWLEDGED`, `CLOSED`, `CANCELLED`
`AnalysisResult`	`AGREED` or `DISAGREED`

The block: only the `InfractionAmount`, not the whole account

This is the part that causes the most bugs if you're not careful. What you block is not the account's entire balance, nor the full amount of the original transaction — it's exactly the InfractionAmount: the amount that node received and that needs to be held on that specific transaction.

Why? Because the money spreads out. A mule account may have received R$ 800 from one branch and R$ 200 from another; each Infraction Report carries the right amount to block on each transaction. Blocking too much holds the customer's legitimate money; blocking too little lets funds slip away. Block the InfractionAmount, no more, no less.

⏱️ The receiving PSP has up to 7 days to analyze and close the report (.../acknowledge on receipt, .../close with AGREED/DISAGREED). Every agreement generates a fraud marker for that receiver — even if they never end up receiving a refund request.

💸 Refund Request (`Refund`) — bringing the money back

Once the Infraction Reports are analyzed (FUNDS_RECOVERY_ANALYSED event), the paying PSP triggers the start of the refund. From there DICT fires the Refund Requests sequentially, without you having to orchestrate them one by one.

The key point: each refund goes to the harmed customer, only for the missing amount

Here lies the biggest mindset shift of 2.0. Every refund request is destined to the harmed customer (the user of the paying PSP who opened the recovery), and the goal is to return the money wherever it is in the chain. Rules that show up on the test:

The 1st request always goes to the receiver of the root transaction.
The subsequent ones follow the priority defined in the blocking stage.
Each PSP debits the mule account and refunds directly to the harmed customer, appearing as the payer on the statement.
Only the missing amount is requested to complete the total — never the full initial amount again. BCB's own example: even with R$ 800 blocked at a node, if R$ 250 was already recovered, the refund is requested for the remaining R$ 750 only. The amounts are updated automatically with each new refund.

In other words: the system keeps "scraping" whatever it can from each account until the total is met, never refunding more than owed.

Main fields

Field	What it is
`TransactionId`	Transaction being refunded
`RefundReason`	`FRAUD`, `OPERATIONAL_FLAW`, `REFUND_CANCELLED`
`RefundAmount`	Requested amount — the remainder, not the full value
`FundsRecoveryId`	(MED 2.0) links the refund to the recovery
`RefundAccount`	(MED 2.0) destination account for the payment (the harmed customer's)
`MonitorAccount`	(MED 2.0) whether the account should be monitored for future blocks
`EffectiveRefundedAmount`	(MED 2.0) on closure, the amount actually refunded
`Status`	`OPEN`, `CLOSED`, `CANCELLED`
`AnalysisResult`	`TOTALLY_ACCEPTED`, `PARTIALLY_ACCEPTED`, `REJECTED`
`RefundRejectionReason`	`NO_BALANCE`, `ACCOUNT_CLOSURE`, `CANNOT_REFUND`, `OTHER`

💡 Partial refund + monitoring is the nastiest real-world case. If the account doesn't have the full balance at the moment (PARTIALLY_ACCEPTED / NO_BALANCE), you accept what you can, report the EffectiveRefundedAmount and, with MonitorAccount on, monitor the account to keep blocking and refunding in parts as money comes in. Model this from the start — it's not an exception, it's routine.

🚩 Fraud Marker (prevention)

MED isn't only about putting out fires — it feeds prevention.

When an Infraction Report is agreed (AGREED), the receiver is flagged as a fraudster in DICT, automatically. And there's an important 2.0 detail: all receivers of transactions with an agreed report get flagged — even the ones that never receive a refund request (e.g., a node that fell outside the continuous path of the graph).

These markers feed DICT's anti-fraud data (counters of settlements, reported frauds and confirmed frauds, by key, by owner and by account).

🎯 Where this becomes gold: query that anti-fraud data for the destination key/account before releasing a cash-out. A recent fraud signal? You hold, ask for a double-check or add friction. Preventing is far cheaper than running the whole recovery flow.

📅 The deadlines you CAN'T miss

A blown deadline in MED = money lost or a regulatory fine. Nail these into your scheduler (the definitive source is the Pix Timing Manual):

Rule	Deadline
Maximum transaction age to open a case	80 days after the Pix
Infraction Report analysis by the receiving PSP	up to 7 days
Start the refund after the reports are analyzed	up to 72 hours

Strong recommendation: model each deadline as a scheduled job with alerts. Don't rely on someone watching a spreadsheet. A due SLA is a system event.

✅ When to use (and NOT use) MED

You can use it for:

Proven scam/fraud
Account takeover / coercion / fraudulent access
The institution's own operational error

You can't use it for:

Commercial disputes ("the product never arrived")
Buyer's remorse
A Pix sent to the wrong person by the payer's own mistake
Disputes between good-faith third parties

🏁 Wrapping up

MED 2.0 changes the game of fraud recovery on Pix: gone is the focus on a single account, in comes tracing the money's path, with targeted blocking and layer-by-layer refunds. For your fintech, the summary is:

🧭 The flow starts with the customer: they open the Funds Recovery, which traces and fires the Infraction Reports.
🔒 Block the InfractionAmount, not the whole account — that's the detail that bites the most.
💸 Each refund goes to the harmed customer and only for the missing amount — the goal is to bring the money back wherever it is.
🔔 Use events, not polling (FUNDS_RECOVERY_ANALYSED, FUNDS_RECOVERY_COMPLETED).
📚 The source of truth is always the DICT API spec and the BCB rules — check the current version before coding.

Official sources (BCB)

Pix rules — Central Bank of Brazil and Resolution BCB nº 493/2025
Circuito Pix — MED 2.0, Day 1 and Day 2
Refund procedures implementation guide (MED)
DICT API — official repository bacen/pix-dict-api

That's it! If you're implementing MED 2.0 at your fintech and hit a wall, drop a comment and let's talk it through. 🚀

Understanding MongoDB $lookup performance

Lucas Aguiar — Fri, 05 Sep 2025 00:30:01 +0000

After running into some issues with $lookup in our banking application, I decided to dive deeper into how it actually works and why it sometimes ends up being so slow.

The problem

It was just a normal day. I was working on a new feature when my tech lead mentioned me on Slack with something like:

“Lucas, your query is too slow”
(and then pasted the Explain Plan)

Right after that, he casually added: “This query took 33 minutes to finish.”

In my head, I was like: “Wow… how did this slip through? The code has been there for three months already!”
That’s when I decided to investigate the problem more deeply.

Here’s the query he sent me:

[
  {
    "$match": "simple start filtering"
  },
  {
    "$lookup": {
      "from": "Company",
      "let": {
        "company": "$company"
      },
      "pipeline": [
        {
          "$match": {
            "removedAt": null,
            "$expr": {
              "$and": [
                {
                  "$eq": [
                    "$_id",
                    "$$company"
                  ]
                }
              ]
            }
          }
        }
      ],
      "as": "company"
    }
  },
  {
    "$lookup": {
      "from": "Customer",
      "let": {
        "customer": "$customer"
      },
      "pipeline": [
        {
          "$match": {
            "removedAt": null,
            "$expr": {
              "$and": [
                {
                  "$eq": [
                    "$_id",
                    "$$customer"
                  ]
                }
              ]
            }
          }
        }
      ],
      "as": "customer"
    }
  },
  {
    "$match": "more filtering"
  },
]

Right after reading the query, my first thought was: “Why are we using $expr and pipelines inside the $lookup?”

So the very first thing I did was paste this aggregation pipeline into MongoDB Compass to run an explain and try to visualize why the query was so slow.
And to my surprise… I just got an infinite loading screen 😅

I promise, I waited a long time hoping to see some output. But in the end, all I got was a timeout error.

At that point, I decided to break things down step by step, isolating each stage to figure out where the bottleneck was hiding.

Testing

First, I ran an explain on just the $match. Everything worked perfectly: no problems at all. The query took only 10ms and used a simple company index.

[
  {
    "$match": {
      "removedAt": null,
      "company": ObjectId("some_ID"),
      "type": "TYPE_FILTERING"
    }
  }
]

So far, so good.

Next, I added the first $lookup and here’s where things started to smell funny. The execution time jumped from 10ms to 314ms.

Before moving on, I decided to test both $lookups together, just to confirm where the problem was. And… boom! 6 minutes to return the result. So clearly, the issue had to be in the $lookups.

Understanding the lookup

Everything here are based on the official documentation of lookup on mongodb.

So, to start, $lookup has this syntax and these fields

{
   $lookup:
     {
       from: <collection to join>,
       localField: <field from the input documents>,
       foreignField: <field from the documents of the "from" collection>,
       let: { <var_1>: <expression>, …, <var_n>: <expression> },
       pipeline: [ <pipeline to run> ],
       as: <output array field>
     }
}

At the basic, the easy way to use an lookup is with localField and foreignField, but on our query we are using let and pipeline to find the results.
But why? At the application we use soft delete to prevent deleting important data by an error, so everything keeps stored but with removedAt flag, but we really needs to filter with pipeline?

Understanding the problem of `$expr`

According to the official $expr documentation:

When $expr appears in a $match stage that is part of a $lookup subpipeline, $expr can refer to let variables defined by the $lookup stage

The problem here is that when MongoDB applies the filter, it has to evaluate the condition document by document inside the collection. As the collection grows, this operation gets slower and slower, since it can’t rely on indexes the same way a direct match does.

In other words: $expr inside a $lookup pipeline forces MongoDB to do more work, which quickly becomes painful at scale.

The better approach is to use localField and foreignField whenever possible. That way, MongoDB can take advantage of indexes efficiently. For example:

  {
    "$lookup": {
      "from": "Company",
      "localField": "company",
      "foreignField": "_id",
      "pipeline": [
        {
          "$match": {
            "removedAt": null,
          }
        }
      ],
      "as": "company"
    }
  },

And here we go! 🎉
Now the response time is totally acceptable:

Improving the pipeline even more

After some experiments, I discovered that whenever we use a pipeline inside a $lookup, the query slows down a lot sometimes 5x to 10x slower.

So, if your application doesn’t really need extra validations, I strongly recommend avoiding the pipeline. Instead, stick to localField/foreignField. This is much simpler and far more efficient when you’re just joining on _ids.

In my case, by removing the unnecessary pipeline, I improved the query from 259ms down to 36ms. Here’s the simplified version:

[
  {
    "$match": "simple filtering"
  },
  {
    "$lookup": {
      "from": "Company",
      "localField": "company",
      "foreignField": "_id",
      "as": "company"
    }
  },
]

Conclusion

After all the tweaks, the final query ended up looking like this:

[
  {
    "$match": "simple filtering"
  },
  {
    "$lookup": {
      "from": "Company",
      "localField": "company",
      "foreignField": "_id",
      "as": "company",
    }
  },
  {
    "$lookup": {
      "from": "Customer",
      "localField": "customer",
      "foreignField": "_id",
      "as": "customer",
    }
  },
  {
    "$match": "more filtering"
  },

And that’s it a (not so short) story, but with some practical tips on how to improve $lookup performance:

✅ Prefer localField / foreignField whenever possible
⚠️ $expr can slow down queries a lot
🛠️ For more complex lookups, combine pipeline with localField/foreignField

Understanding IMAP and SMTP Mail Protocols

Lucas Aguiar — Sat, 26 Jul 2025 01:48:38 +0000

During a recent implementation of a customer service platform, I encountered several configuration challenges related to email communication protocols. This experience prompted me to conduct thorough research into SMTP and IMAP protocols, their interactions, and best implementation practices. The knowledge gained proved invaluable for resolving our technical issues, and I'm sharing these insights to benefit others facing similar challenges in their systems architecture.

📮 Introduction

In the digital world, sending and receiving emails are fundamental communication processes. During my recent experience, I realized that understanding how these protocols work behind the scenes can make all the difference, especially when configuring or troubleshooting customer service systems.

🌐 Overview of Email Protocols

Emails travel between computers using standardized protocols. The most common ones are:

SMTP (Simple Mail Transfer Protocol): Used for sending messages
IMAP (Internet Message Access Protocol): Used for accessing and managing messages stored on the server

While POP3 also exists, this article focuses on SMTP and IMAP, which offer advantages for synchronization and email management across multiple devices.

📤 SMTP - Simple Mail Transfer Protocol

SMTP is the "engine" that sends emails. Here's a summary of what I learned:

Primary Function: SMTP is responsible for sending messages from the sender to the server, and between servers, ensuring the email reaches its recipient.
How It Works:
- The process begins with the email client sending the message to the SMTP server
- Commands like EHLO/HELO, MAIL FROM, RCPT TO, and DATA structure this communication
Ports and Security:
- Common ports include 25 (traditional) and 587 (for authenticated sending)
- Using TLS or SSL encryption is highly recommended to protect data during transmission

📥 IMAP - Internet Message Access Protocol

While SMTP handles sending, IMAP is the "manager" of received emails:

Primary Function: IMAP allows you to access and manage emails directly on the server. This means that regardless of which device you use—smartphone, tablet, or computer—your messages remain synchronized.
Advantages over POP3:
- Messages remain stored on the server, allowing viewing and organization from any device
- You can create, move, and delete folders directly on the server without downloading emails
How It Works:
- IMAP synchronizes message status (e.g., read/unread) and organizes emails into folders
Ports and Security:
- Port 143 is used for unencrypted connections, while port 993 is for secure connections via SSL/TLS

⚖️ Comparison Between SMTP and IMAP

Although both protocols are part of email communication, they have distinct, complementary roles:

SMTP:
- Responsible for sending emails
- Functions as the "digital mail carrier" distributing messages between servers
IMAP:
- Responsible for accessing and managing stored messages
- Allows you to track your emails in a synchronized manner across multiple devices

⚙️ Technical Aspects and Implementation

During my experience, I found that visualizing the email flow helps better understand how these protocols interact:

Sending and Receiving Flow:
- The email leaves the client and goes to the SMTP server
- From there, it may be forwarded to other servers until it reaches the recipient's server
- Finally, the email client uses IMAP to access and manage this message on the server
Configuration and Authentication:
- Correctly configuring ports and authentication methods (such as TLS/SSL) is essential to ensure everything works smoothly
- In many situations, the challenge lies precisely in aligning these configurations for efficient and secure customer service operation

🔒 Security and Best Practices

To avoid headaches (like the one I faced), it's important to pay attention to these points:

Common Threats:
- Without proper security, data can be intercepted during transmission
- Configuration errors can create vulnerabilities for spam or even more serious attacks
Security Measures:
- Always use encryption (SSL/TLS) in communications
- Regularly verify and update server authentication configurations
- Follow recommended best practices for both administrators and end users

✅ Conclusion

Understanding SMTP and IMAP protocols was essential to solving the problem I encountered when configuring the customer service system. With SMTP handling sending and IMAP enabling access and message management, both complement each other to make email communication fluid and secure.

I hope this article, written simply and based on my experience, helps you better understand how these protocols work. After all, sharing knowledge is a way to grow and make life easier for those who also need to deal with these configurations daily.

If you have questions or want to explore any topic in greater depth, feel free to comment. We're all learning together!

Prometheus + Redis: Simple Guide to Monitor Redis Instances

Lucas Aguiar — Mon, 21 Apr 2025 22:26:33 +0000

Redis is often a critical part of modern infrastructure — whether used as a cache, message broker, or ephemeral store. Monitoring it properly helps you detect issues early and ensure system stability.

In this guide, you’ll learn how to build a Redis monitoring dashboard with Prometheus and Grafana to quickly identify issues and improve system reliability.

⚙️ Installing Prometheus (Quick Setup)

We’ll start by deploying Prometheus using Helm:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm install kube-prometheus prometheus-community/kube-prometheus-stack \
  --namespace prometheus --create-namespace

This will install Prometheus, Grafana, and some built-in exporters.
To access Grafana:

kubectl port-forward svc/kube-prometheus-grafana -n prometheus 3000:80

You can access here -> http://localhost:3000/login

Default login: admin / prom-operator

🧱 Installing Redis

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update

helm install redis bitnami/redis \
  --namespace app-stg \
  --set architecture=standalone \
  --set auth.enabled=false \
  --create-namespace

This installs a standalone Redis with no password, perfect for testing.
After this, you can point the Redis Exporter and your application to the correct Redis service. The hostname will be:

redis-master.app-stg.svc.cluster.local:6379

📦 Installing Redis Exporter

Now let’s deploy the Redis Exporter:

helm install redis-exporter prometheus-community/prometheus-redis-exporter \
  --namespace prometheus \
  --set serviceMonitor.enabled=true \
  --set serviceMonitor.namespace=prometheus \
  --set serviceMonitor.interval=15s \
  --set serviceMonitor.labels.release=kube-prometheus \
  --set redisAddress=redis://redis-master.app-stg.svc.cluster.local:6379 \
  --create-namespace

The Redis Exporter exposes metrics at a Prometheus-compatible /metrics endpoint.

kubectl port-forward svc/redis-exporter-prometheus-redis-exporter -n prometheus 9121:9121

Then open: http://localhost:9121/metrics
You should see a long list of metrics like this:

✅ redis_up 1 means the exporter is connected to a Redis instance and it’s healthy.
🚫 redis_up 0 means the exporter failed to connect — check the Redis address or service availability.

If everything is working, you’ll also see metrics like:

redis_connected_clients
redis_memory_used_bytes
redis_total_commands_processed

🔍 Checking in Prometheus

To validate that Prometheus is collecting Redis metrics:

kubectl port-forward svc/kube-prometheus-kube-prome-prometheus -n prometheus 9090

Open http://localhost:9090, go to Status > Targets, and check that Redis Exporter is listed and UP.

Everything is working right!

📊 Importing the Redis Dashboard on Grafana

To visualize your Redis metrics in a clean and ready-to-use panel, you can import a community-maintained dashboard from Grafana Labs.

https://grafana.com/grafana/dashboards/14091-redis-dashboard-for-prometheus-redis-exporter-1-x/

🧭 How to Import

Access Grafana by establishing port forwarding and navigating to http://localhost:3000 in your browser
Navigate to the "+" icon in the left sidebar and select "Import"
Enter the dashboard ID or URL in the provided field and click "Load"
Click "Import" to finalize the installation

Allow a few moments for the system to process your request as the metrics populate on your dashboard

Thats it!
⎈Happy Helming!⎈

Explorando um Cluster MongoDB com Terraform

Lucas Aguiar — Sun, 02 Feb 2025 02:12:41 +0000

A pouco tempo após algumas conversas me interessei por aprender mais sobre terraform e como provisionar infraestrutura de maneira mais fácil, simples de replicar em outros lugares e manter a longo prazo.

A ideia aqui é demonstrar como a abordagem de Infrastructure as Code pode simplificar a criação de um cluster MongoDB, composto por um nó primário e dois nós secundários. Utilizando o Terraform e o Docker como provedor para teste local.

O passo a passo

1. Preparando o ambiente

Primeiro, vamos definir o docker como provedor e uma rede para que os containers se comuniquem usando a resolução de DNS interno, podendo usar "mongo_primary", "mongo_secondary_1" e "mongo_secondary_2" apontando para os ips internos corretos.

terraform {
  required_providers {
    docker = {
      source  = "kreuzwerker/docker"
      version = "~> 3.0.2"
    }
  }
}

provider "docker" {}

resource "docker_network" "mongo_network" {
  name = "mongo_network"
}

2. Gerando os arquivos internos de autenticação

Antes de subir os containers, é necessário gerar um arquivo keyfile que será utilizado para a autenticação interna entre os nós do replica set do MongoDB. Esse arquivo garante que somente os nós autorizados possam se comunicar.

openssl rand -base64 756 > mongo-keyfile

Alterando as permissões para que somente o proprietário possa ler o arquivo

chmod 400 mongo-keyfile

Certifique-se de que o arquivo mongo-keyfile esteja na mesma pasta onde se encontra o arquivo main.tf ( ou em uma pasta no mesmo nível ), pois ele será referenciado nos volumes dos containers.

3. Configurando o replicaset

Separamos aqui os containers em duas partes 1 nó primário e 2 secundários, em um cluster replica set de mongodb o nó primário é responsável por aceitar operações de escrita e propagar as alterações para os secundários, que mantêm uma cópia dos dados e podem ser utilizados para balanceamento de carga em leituras.

// init-replica.js

conn = new Mongo("mongodb://admin:adminpass@localhost:27017");
db = conn.getDB("admin");

try {
    let status = rs.status();
    print("ℹ️ Replica Set já configurado.");
} catch (err) {
    if (err.codeName === "NotYetInitialized") {
        print("🚀 Iniciando Replica Set...");
        rs.initiate({
            _id: "rs0",
            members: [
                { _id: 0, host: "mongo_primary:27017" },
                { _id: 1, host: "mongo_secondary_1:27017" },
                { _id: 2, host: "mongo_secondary_2:27017" }
            ]
        });
        print("✅ Replica Set inicializado com sucesso!");
    } else {
        print("❌ Erro ao verificar status do Replica Set:", err);
        throw err;
    }
}

4. Subindo os containers mongodb

No terraform a configuração dos containers mongodb é similar, entretanto, só o nó primário recebe instruções de exposição das portas e o script de configuração do cluster.

Montamos os dois volumes referente ao keyfile de autenticação e o script de configuração de replica em js.

resource "docker_container" "mongo_primary" {
  name    = "mongo_primary"
  image   = "mongo:7.0"
  restart = "always"

  networks_advanced {
    name = docker_network.mongo_network.name
  }

  ports {
    internal = 27017
    external = 27017
  }

  volumes {
    host_path      = "${abspath(path.module)}/mongo-keyfile"
    container_path = "/data/keyfile"
  }

  volumes {
    host_path      = "${abspath(path.module)}/init-replica.js"
    container_path = "/docker-entrypoint-initdb.d/init-replica.js"
  }

  env = [
    "MONGO_INITDB_ROOT_USERNAME=admin",
    "MONGO_INITDB_ROOT_PASSWORD=adminpass",
    "MONGO_REPLICA_SET_NAME=rs0"
  ]

  command = [
    "mongod",
    "--replSet", "rs0",
    "--keyFile", "/data/keyfile",
    "--bind_ip_all"
  ]

  provisioner "local-exec" {
    command = "sleep 10 && docker exec mongo_primary mongosh /docker-entrypoint-initdb.d/init-replica.js"
  }
}

resource "docker_container" "mongo_secondary_1" {
  name  = "mongo_secondary_1"
  image = "mongo:7.0"
  restart = "always"

  networks_advanced {
    name = docker_network.mongo_network.name
  }

  volumes {
    host_path      = "${abspath(path.module)}/mongo-keyfile"
    container_path = "/data/keyfile"
  }

  env = [
    "MONGO_INITDB_ROOT_USERNAME=admin",
    "MONGO_INITDB_ROOT_PASSWORD=adminpass",
    "MONGO_REPLICA_SET_NAME=rs0"
  ]

  command = [
    "mongod",
    "--replSet", "rs0",
    "--keyFile", "/data/keyfile",
    "--bind_ip_all"
  ]
}

resource "docker_container" "mongo_secondary_2" {
  name  = "mongo_secondary_2"
  image = "mongo:7.0"
  restart = "always"

  networks_advanced {
    name = docker_network.mongo_network.name
  }

  volumes {
    host_path      = "${abspath(path.module)}/mongo-keyfile"
    container_path = "/data/keyfile"
  }

  env = [
    "MONGO_INITDB_ROOT_USERNAME=admin",
    "MONGO_INITDB_ROOT_PASSWORD=adminpass",
    "MONGO_REPLICA_SET_NAME=rs0"
  ]

  command = [
    "mongod",
    "--replSet", "rs0",
    "--keyFile", "/data/keyfile",
    "--bind_ip_all"
  ]
}

5. Integração com aplicações

Uma vez com o cluster configurado, é possível se conectar a ele através do comando:

docker exec -it mongo_primary mongosh "mongodb://admin:adminpass@localhost:27017/?replicaSet=rs0"

Para que uma aplicação tenha acesso ao cluster, é necessário que ela esteja junto a "mongo_network" para que não haja problemas na resolução de DNS.

Conclusão

Este exemplo demonstra como o terraform pode ser uma ferramenta poderosa para criar ambientes consistentes e fáceis de serem replicados, lembrando que o exemplo anterior não é recomendado para uso em produção pois cada nó deve estar em uma máquina separada para garantir alta disponibilidade.

Caso tenha ficado algo confuso, você pode acompanhar meu código no Github:

https://github.com/lusqua/terraform-mongodb-cluster

Desvendando a VPC na AWS: Subnets, Gateways e Configuração de Rede

Lucas Aguiar — Thu, 31 Oct 2024 10:30:00 +0000

Ontem à noite, após o expediente, decidi resolver alguns problemas técnicos em um side project na AWS. O desafio? Criar uma infraestrutura robusta, com subnets públicas e privadas, garantindo que apenas os recursos certos tivessem acesso à internet. A ideia era entender de uma vez por todas como cada peça se encaixa: Internet Gateway, NAT Gateway e os detalhes de roteamento.

O que parecia simples se revelou mais complexo. Ao configurar meu Auto Scaling Group, notei que as instâncias EC2 não estavam acessando a internet, mesmo quando atribuídas a subnets públicas. Isso me fez mergulhar fundo nas configurações de roteamento, nas permissões de IP público e nas nuances da AWS.

Aqui estão as lições e conceitos principais que consegui extrair desse sofrimento desenfreado:

Subnet Pública vs. Privada

Para simplificar, uma subnet pública permite que instâncias tenham acesso direto à internet através de um Internet Gateway (IGW), essencial para servidores web ou APIs externas. No entanto, minha aplicação precisava de segurança adicional, então configurei uma subnet privada para meus recursos internos, como bancos de dados. Essas subnets privadas não têm acesso direto à internet, mas, com um NAT Gateway, consegui permitir que os recursos façam atualizações ou se comuniquem externamente quando necessário.

Configurando o Internet Gateway e o NAT Gateway

Para as subnets públicas, configurei um Internet Gateway e adicionei uma rota que apontava 0.0.0.0/0 para ele, o que garantiu que as instâncias pudessem enviar e receber tráfego da internet. Já para as subnets privadas, configurei um NAT Gateway na subnet pública, permitindo que o tráfego externo fosse roteado indiretamente. Isso evitou a exposição direta à internet, mas ainda permitiu que as instâncias privadas acessem atualizações externas.

Desafios com o Auto Scaling e Distribuição de Tarefas

Um dos problemas mais complexos foi garantir que as tarefas do ECS fossem distribuídas entre as instâncias EC2 de maneira balanceada. Inicialmente, minhas tarefas não se espalhavam como esperado, então experimentei diferentes estratégias de Task Placement, como “AZ balanced binpack”, que ajudou a otimizar o uso das instâncias e garantir redundância entre as zonas de disponibilidade.

Após uma noite inteira, finalmente consegui configurar uma arquitetura estável, com recursos bem distribuídos e segura. Este side project foi uma experiência reveladora, me dando não apenas o domínio sobre subnets e gateways na AWS, mas também uma lição sobre resiliência e atenção aos detalhes.

Conclusão

O processo de configurar subnets públicas e privadas, Internet Gateway, e NAT Gateway pode ser complicado, mas é essencial para arquiteturas seguras e escaláveis. Para quem enfrenta os mesmos desafios, espero que este artigo forneça uma visão prática e inspiradora para perseverar, mesmo que isso signifique uma longa noite de trabalho.

Automatizando deployments com WatchTower e GitHub Container Registry

Lucas Aguiar — Tue, 08 Oct 2024 17:05:55 +0000

Manter aplicações atualizadas é crucial para garantir segurança, performance e a incorporação de novas funcionalidades. No mundo dos containers Docker, WatchTower surge como uma ferramenta poderosa para automatizar esse processo. Neste artigo, vamos explorar como integrar o WatchTower com o GitHub Container Registry (GHCR), criando um fluxo contínuo de atualização para seus containers. Vamos abordar desde a configuração inicial até um exemplo prático para ilustrar todo o processo.

🚀 Introdução ao WatchTower e GHCR

O que é o WatchTower?

WatchTower é uma ferramenta de código aberto que monitora seus containers Docker e automaticamente os atualiza quando uma nova imagem é detectada no registry configurado. Isso elimina a necessidade de intervenções manuais para manter suas aplicações sempre atualizadas com as últimas versões.

O que é o GitHub Container Registry (GHCR)?

O GitHub Container Registry é um serviço de registro de containers integrado ao GitHub, permitindo que você armazene e gerencie suas imagens Docker diretamente nos repositórios do GitHub. Ele oferece integração perfeita com workflows de CI/CD, facilitando a publicação e o gerenciamento de imagens.

🛠️ Pré-requisitos

Antes de começarmos, certifique-se de ter o seguinte:

Docker instalado na sua máquina ou servidor.
Conta no GitHub com acesso ao GitHub Container Registry.
Repositório GitHub para hospedar sua aplicação e workflow.
Conhecimentos básicos de Docker e GitHub Actions.

🔧 Passo a Passo: Integrando WatchTower com GHCR

Configurando o GitHub Container Registry

Primeiro, vamos configurar o GHCR para armazenar nossas imagens Docker.

a. Autenticação no GHCR

Para autenticar o Docker com o GHCR, você precisa de um token de acesso pessoal com permissão para write:packages e read:packages.

echo $CR_PAT | docker login ghcr.io -u USERNAME --password-stdin

Substitua CR_PAT pelo seu token de acesso pessoal e USERNAME pelo seu nome de usuário do GitHub.

b. Criando uma Imagem Docker

Vamos criar uma imagem Docker simples para este exemplo. Crie um arquivo Dockerfile:

# Dockerfile
FROM node:14-alpine
WORKDIR /app
COPY package.json .
RUN npm install
COPY . .
EXPOSE 3000
CMD ["npm", "start"]

E um arquivo package.json básico:

{
  "name": "watchtower-example",
  "version": "1.0.0",
  "main": "index.js",
  "scripts": {
    "start": "node index.js"
  },
  "dependencies": {
    "express": "^4.17.1"
  }
}

Crie também um index.js simples:

const express = require('express');
const app = express();
const port = 3000;

app.get('/', (req, res) => {
  res.send('Hello from WatchTower and GHCR!');
});

app.listen(port, () => {
  console.log(`App running on port ${port}`);
});

c. Construindo e Publicando a Imagem

Construa a imagem Docker e publique-a no GHCR.

docker build -t ghcr.io/USERNAME/watchtower-example:latest .
docker push ghcr.io/USERNAME/watchtower-example:latest

Substitua USERNAME pelo seu nome de usuário do GitHub.

Configurando o WatchTower

Agora, vamos configurar o WatchTower para monitorar o GHCR e atualizar automaticamente nossos containers.

a. Criando um Arquivo de Configuração do WatchTower

Crie um arquivo docker-compose.yml para orquestrar o WatchTower e sua aplicação.

version: '3'

services:
  app:
    image: ghcr.io/USERNAME/watchtower-example:latest
    container_name: watchtower-example
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production

  watchtower:
    image: containrrr/watchtower
    container_name: watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - WATCHTOWER_CLEANUP=true
      - WATCHTOWER_POLL_INTERVAL=300
    command: --registry ghcr.io --interval 300

Explicações:

app: Serviço da aplicação que será monitorada pelo WatchTower.
watchtower: Serviço do WatchTower configurado para monitorar o GHCR com um intervalo de verificação de 5 minutos (300 segundos).

b. Variáveis de Ambiente para Autenticação

Para que o WatchTower possa acessar o GHCR privado, precisamos fornecer as credenciais. Atualize o serviço watchtower no docker-compose.yml:

environment:
  - WATCHTOWER_CLEANUP=true
  - WATCHTOWER_POLL_INTERVAL=300
  - WATCHTOWER_REGISTRY_AUTH=true
  - WATCHTOWER_REGISTRY_USERNAME=USERNAME
  - WATCHTOWER_REGISTRY_PASSWORD=CR_PAT

Substitua USERNAME pelo seu nome de usuário do GitHub e CR_PAT pelo seu token de acesso pessoal.

Automatizando a Atualização com GitHub Actions

Para garantir que sempre que você fizer push no repositório, a imagem Docker seja atualizada no GHCR, vamos configurar um workflow no GitHub Actions.

Crie um arquivo .github/workflows/docker-publish.yml no seu repositório:

name: Publish Docker image

on:
  push:
    branches: [ main ]

jobs:
  build-and-push:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to GHCR
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push
        uses: docker/build-push-action@v4
        with:
          context: .
          push: true
          tags: ghcr.io/USERNAME/watchtower-example:latest

Explicações:

actions/checkout@v3: Faz o checkout do código.
docker/setup-buildx-action@v3: Configura o Docker Buildx.
docker/login-action@v3: Autentica no GHCR usando o token do GitHub.
docker/build-push-action@v4: Constrói e publica a imagem no GHCR.

Substitua USERNAME pelo seu nome de usuário do GitHub.

Implementando e Testando

a. Deploy Inicial

Inicie os serviços usando o Docker Compose:

docker-compose up -d

Isso iniciará sua aplicação e o WatchTower. O WatchTower começará a monitorar o GHCR a cada 5 minutos (300 segundos) para verificar se há novas imagens disponíveis.

b. Atualizando a Aplicação

Faça uma alteração no código da sua aplicação, por exemplo, atualize a mensagem em index.js:

res.send('Hello from WatchTower, GHCR, and your updated app!');

Commit e push as alterações para o repositório GitHub:

git add .
git commit -m "Update welcome message"
git push origin main

O GitHub Actions irá construir e publicar a nova imagem no GHCR. Após até 5 minutos, o WatchTower detectará a nova imagem e atualizará automaticamente o container watchtower-example sem downtime.

🎉 Conclusão

Integrar o WatchTower com o GitHub Container Registry oferece uma solução eficiente para manter suas aplicações Docker sempre atualizadas de forma automática. Essa automação não apenas economiza tempo, mas também garante que suas aplicações estejam sempre rodando as versões mais recentes e seguras. Além disso, combinando com ferramentas como o Traefik, você pode criar uma infraestrutura robusta e escalável, pronta para atender às demandas de ambientes de produção modernos.

Experimente implementar essa integração no seu projeto e aproveite os benefícios da automação contínua!

📚 Recursos Adicionais

Implementing DataLoader and Understanding Its Advantages Over Lookup

Lucas Aguiar — Thu, 15 Aug 2024 00:48:20 +0000

If you’ve been working with databases, especially in a GraphQL environment, you might have encountered the "N+1 problem." This issue occurs when your system makes multiple database calls to fetch related data, which can significantly slow down your application. Enter DataLoader—a tool that’s here to save the day by batching and caching database requests, making your queries much more efficient.

Let’s dive into how DataLoader works and see it in action with a practical example involving bank transactions and user details.

What is DataLoader?

DataLoader is a utility designed to batch and cache requests for data. Instead of making multiple calls to fetch each piece of data separately, DataLoader groups these requests together, sends them in one go, and then caches the results for future use. This approach is particularly useful in reducing the number of database queries, thus speeding up your application.

How DataLoader Solves the N+1 Problem

Consider a scenario where you have a list of banking transactions, and each transaction is associated with a user. Without DataLoader, you might end up querying the database for each user separately, leading to multiple (and often redundant) database queries. This is where DataLoader shines—it batches these requests together, so the database is queried only once per user.

Implementing DataLoader: A Practical Example

Let’s imagine you’re building a banking application, and you need to list transactions along with the details of the users who made those transactions. Here’s how you can implement DataLoader to optimize this process:

Setting Up the DataLoader:

   import DataLoader from 'dataloader';
   import { ObjectId } from 'mongodb';
   import { getCollection } from './mongodbConfig';  // Your MongoDB configuration file

   // Batch function to fetch users by IDs
   export const usersInBatch = async (ids) => {
     const coll = await getCollection('users');
     const objIdIds = ids.map(id => new ObjectId(id));
     const found = await coll.find({ _id: { $in: objIdIds } }).toArray();
     return found;
   };

   // Create a DataLoader instance for users
   export const userLoader = new DataLoader(usersInBatch);

Here, usersInBatch is a batch loading function that fetches user details for multiple userIds at once, and userLoader is the DataLoader instance that leverages this function.

Using DataLoader in Your Application:

   import { userLoader } from './dataLoaderConfig';  // Your DataLoader configuration file

   // Function to get transaction details
   export const getTransactionDetails = async (transaction) => {
     const user = await userLoader.load(transaction.userId);
     return {
       ...transaction,
       userName: user.name,
       userEmail: user.email,
     };
   };

   // Function to list transactions with user details
   export const listTransactions = async (page = 1, limit = 20) => {
     const coll = await getCollection('transactions');
     const skip = (page - 1) * limit;
     const rawTransactions = await coll.find().skip(skip).limit(limit).toArray();

     return await Promise.all(rawTransactions.map(getTransactionDetails));
   };

In this setup:

The getTransactionDetails function uses userLoader to fetch user details by userId, ensuring that each user is only fetched once per batch.
listTransactions retrieves a page of transactions and maps over them to enrich each transaction with the relevant user details.

Why Use DataLoader?

By using DataLoader, you reduce the number of database queries from potentially hundreds (or more) to just a few. This not only optimizes performance but also reduces load on your database, which can be crucial in high-traffic applications. Moreover, DataLoader’s built-in caching mechanism ensures that once a user’s data is fetched, it doesn’t need to be fetched again in the same request cycle, further speeding up your application.

Conclusion

DataLoader is an invaluable tool when dealing with scenarios that involve repeated database lookups, particularly in applications with complex data relationships like the one we explored here. By batching and caching requests, it dramatically improves the efficiency and performance of your database interactions. Whether you’re dealing with banking transactions, product orders, or any other domain where related data is frequently accessed, DataLoader can help you keep your application running smoothly and efficiently.

If you're facing performance issues due to repeated database queries, give DataLoader a try—your application (and your users) will thank you!

DEV Community: Lucas Aguiar

MED 2.0 for devs: entities, fields and the implementation flow

🧩 First: what MED is (and why it exists)

🚀 What changes in MED 2.0

🧱 The MED entities (and how they connect)

🕸️ Funds Recovery (FundsRecovery)

The stages (this is where the "several statuses" live)

Tracking graph parameters

Main endpoints

📢 Infraction Report (InfractionReport) — the block

Main fields

The block: only the InfractionAmount, not the whole account

💸 Refund Request (Refund) — bringing the money back

The key point: each refund goes to the harmed customer, only for the missing amount

Main fields

🚩 Fraud Marker (prevention)

📅 The deadlines you CAN'T miss

✅ When to use (and NOT use) MED

🏁 Wrapping up

Official sources (BCB)

Understanding MongoDB $lookup performance

The problem

Testing

Understanding the lookup

Understanding the problem of $expr

Improving the pipeline even more

Conclusion

Understanding IMAP and SMTP Mail Protocols

📮 Introduction

🌐 Overview of Email Protocols

📤 SMTP - Simple Mail Transfer Protocol

📥 IMAP - Internet Message Access Protocol

⚖️ Comparison Between SMTP and IMAP

⚙️ Technical Aspects and Implementation

🔒 Security and Best Practices

✅ Conclusion

Prometheus + Redis: Simple Guide to Monitor Redis Instances

⚙️ Installing Prometheus (Quick Setup)

🧱 Installing Redis

📦 Installing Redis Exporter

🔍 Checking in Prometheus

📊 Importing the Redis Dashboard on Grafana

🧭 How to Import

Explorando um Cluster MongoDB com Terraform

O passo a passo

1. Preparando o ambiente

2. Gerando os arquivos internos de autenticação

3. Configurando o replicaset

4. Subindo os containers mongodb

5. Integração com aplicações

Conclusão

Desvendando a VPC na AWS: Subnets, Gateways e Configuração de Rede

Subnet Pública vs. Privada

Configurando o Internet Gateway e o NAT Gateway

Desafios com o Auto Scaling e Distribuição de Tarefas

Conclusão

Automatizando deployments com WatchTower e GitHub Container Registry

🚀 Introdução ao WatchTower e GHCR

🛠️ Pré-requisitos

Implementing DataLoader and Understanding Its Advantages Over Lookup

What is DataLoader?

How DataLoader Solves the N+1 Problem

Implementing DataLoader: A Practical Example

Why Use DataLoader?

Conclusion

🕸️ Funds Recovery (`FundsRecovery`)

📢 Infraction Report (`InfractionReport`) — the block

The block: only the `InfractionAmount`, not the whole account

💸 Refund Request (`Refund`) — bringing the money back

Understanding the problem of `$expr`