DEV Community

Cover image for Learn How to Query from Unlabeled Data Streams in Federated Learning.
Ndukwe Daniel
Ndukwe Daniel

Posted on

Learn How to Query from Unlabeled Data Streams in Federated Learning.

1. The Problem

The authors start from a realistic problem.

Most Federated Learning research assumes that clients already have a labeled dataset.

But what happens when data is continuously arriving at the client without labels?

For example:

Hospital
   ↓
Patient image arrives
   ↓
No label yet
   ↓
Doctor / expert must annotate it
   ↓
Then it can be used for training
Enter fullscreen mode Exit fullscreen mode

Imagine a client continuously receiving new data.

The problem is that getting an expert to label every single sample is expensive.

So you can't simply label everything.

This creates the question:

Which incoming samples should the client spend its labeling budget on?

This is the data-querying problem that the paper is trying to solve.


2. The Deeper Problem

Now, a deeper problem arises.

Suppose Client 1 receives:

A, B, C, D, E

But it only has enough budget to label 2 samples.

The client might think:

"A and B are the most useful for my local model."

That sounds reasonable.

But there is a problem.

Federated Learning is not trying to optimize only Client 1's model.

It is trying to optimize the global model across all clients.

So the samples that are most useful from Client 1's local perspective might not necessarily be the samples that are most useful for the global training objective.

The authors describe this conflict between the local data-access perspective and the global training objective as a fundamental challenge.

And this is the key problem that LeaDQ tries to attack.


3. The System Architecture

                    SERVER
                      ↓
                 Global Model
                      ↓
          ┌───────────┴───────────┐
          ↓                       ↓
      Hospital 1              Hospital K
          ↓                       ↓
 Unlabeled images          Unlabeled images
          ↓                       ↓
    Query policy            Query policy
          ↓                       ↓
   Select samples           Select samples
          ↓                       ↓
  Doctor / expert          Doctor / expert
      labels                    labels
          ↓                       ↓
   Local dataset            Local dataset
          ↓                       ↓
   Local training           Local training
          └───────────┬───────────┘
                      ↓
                 SERVER
                      ↓
              Global Model
Enter fullscreen mode Exit fullscreen mode

The overall process has two main phases.

Phase 1 — Data Querying

Unlabeled data
   ↓
Selected Samples
   ↓
Obtain labels
Enter fullscreen mode Exit fullscreen mode

Clients receive incoming unlabeled data.

The querying policy decides which samples are worth sending to an oracle for labeling.

Phase 2 — Federated Training

So the overall idea is basically:

Unlabeled data
   ↓
Selected Sample
   ↓
Get labels
   ↓
FedAvg
   ↓
New Global model
   ↓
Repeat
Enter fullscreen mode Exit fullscreen mode

The selected samples are labeled and then used for federated training.

After training, more unlabeled data arrives and the process repeats.


4. How Does the Client Decide What to Select?

This is where LeaDQ becomes a reinforcement learning problem.

For each incoming sample, the policy makes a binary decision.

1 = query this sample

0 = don't query this sample
Enter fullscreen mode Exit fullscreen mode

The decision is basically:

Select the sample?

  • Yes → query the oracle for its label
  • No → don't query it

So if 5 samples arrive:

x1 x2 x3 x4 x5
Enter fullscreen mode Exit fullscreen mode

The policy might make decisions like:

1 0 1 0 0
Enter fullscreen mode Exit fullscreen mode

and then:

x1 and x3
Enter fullscreen mode Exit fullscreen mode

Are then sent to the oracle for labeling.

But there is an important constraint.

The client only has a limited querying budget.

So:

Number of selected samples = Nq

The policy therefore has to decide which samples are worth spending that limited budget on.


5. What Information Does the Query Policy See?

The policy needs some information before deciding whether to select an incoming sample.

The observation it receives includes the model's predictive logits.

Compute Predicitive logits
   ↓
local Observation
   ↓
Query Policy
   ↓
Q-Values
   ↓
Choose Samples
Enter fullscreen mode Exit fullscreen mode

So the policy gets information about:

"How does the current model respond to this incoming data?"

For example, if the model is very uncertain about a sample, that information can potentially influence the querying decision.

The important thing here is that the policy isn't simply looking at the raw sample and randomly deciding whether to select it.

It is using information from the current model's response to the incoming data.


6. Why Reinforcement Learning?

This is probably the part that made the most sense to me once I understood the problem.

The client doesn't immediately know:

"Was choosing this sample a good decision?"

For example:

Client Selects:[](
![ ](https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/7yh6durj4nysv3dun554.png))

Sample X
   ↓
Get labels
   ↓
Trains
   ↓
Global model Change
   ↓
Global Accuracy Improves
Enter fullscreen mode Exit fullscreen mode

Suppose the client chooses sample A.

At that exact moment, it doesn't know whether A will actually help the global model.

The consequence of that decision might only become visible after the sample is labeled and used during federated training.

So there is a delay between:

Decision → Training → Global model performance

That is exactly the kind of problem reinforcement learning can model.

The querying policy takes an action, and later receives information about the consequences of that action.

The authors therefore formulate the querying problem as a:

Decentralized Partially Observable Markov Decision Process (Dec-POMDP).


7. Why "Partially Observable"?

This part is important.

Each client only sees its own information.

Client 1 doesn't know exactly what data Client 2 has.

Client 2 doesn't know exactly what Client 3 has.

And so on.

But the objective they're ultimately trying to optimize depends on the global federated learning system.

So each client only has a partial view of the overall environment.

Hence:

Decentralized + Partially Observable → Dec-POMDP

Decentralized 
+
Partially Observable
+
Multiple Decision-Makers
Enter fullscreen mode Exit fullscreen mode

8. They Use MARL

Now we have another problem.

There isn't just one client making decisions.

There are multiple clients making querying decisions.

So the authors use:

Multi-Agent Reinforcement Learning (MARL).

Each client can be treated as an agent.

Client 1  → Policy y1
Client 2  → Policy y2

Client K  → Policy yk
Enter fullscreen mode Exit fullscreen mode

The policies make their decisions locally.

So Client 1 has its own policy.

Client 2 has its own policy.

Client 3 has its own policy.

And they are all making decisions based on their own local observations.


9. But How Do Local Policies Learn About the Global Objective?

This is where Centralized Training with Decentralized Execution (CTDE) comes in.

The basic idea is:

During training

The system can use global information to help train the policies.

During actual operation

Each client has to make decisions using only its local information.

So:

Training can be centralized, but execution remains decentralized.

This makes sense for Federated Learning because we don't want every client to require access to all the other clients' private data just to make a querying decision.


10. Their Biggest Technical Component: QMIX

The authors adapt QMIX for the multi-agent policy training.

Each client has its own local Q-function:

Q₁, Q₂, Q₃, ..., Qₖ

These local Q-functions represent the value of actions from the perspective of each individual agent.

But the overall goal is still the global objective.

QMIX is used to combine these individual value functions into a global value function.

Individual client decisions

↓

Individual Q-functions

↓

QMIX

↓

Global Q-value
Enter fullscreen mode Exit fullscreen mode

The important idea is that the clients can learn policies that make local decisions while still being trained toward a shared global objective.


11. What I Think Is Interesting About This Paper

What I found interesting is that the paper doesn't treat data selection as simply:

"Which samples are the most useful to this client?"

Instead, it asks a much harder question:

"Which samples should this client query so that they ultimately help the global Federated Learning objective?"

That difference is important.

A sample can be very useful for one client's local model but not necessarily useful for the overall federated system.

LeaDQ tries to bridge that gap by treating the querying process as a multi-agent decision-making problem.


12. Limitations

The approach is interesting, but it also introduces quite a lot of complexity.

Instead of simply running Federated Learning, we now have:

Data querying

Reinforcement Learning

Multi-Agent Reinforcement Learning

Dec-POMDP

CTDE

QMIX

Federated Training

That is a lot of machinery around the original problem of deciding which data to label.

There is also the practical question of how well these policies can be trained and how much additional computation and coordination the approach requires.

So while the idea is interesting, implementing something like LeaDQ is definitely more complicated than simply running a standard FedAvg system.


Final Thoughts

The biggest thing I took away from this paper is that data selection in Federated Learning is not just a local optimization problem.

When data is unlabeled and labeling is expensive, the client has to decide where to spend its limited labeling budget.

But the client's decision should ultimately help the global model, not just its own local model.

LeaDQ approaches this by turning the data-querying problem into a multi-agent reinforcement learning problem, where clients make decisions locally while being trained toward a global objective.

And that was my main takeaway from Paper 5.

There is a lot more technical detail in the paper, especially around the QMIX formulation and how the querying policy is trained, but this was my attempt to understand the core idea behind the work before diving deeper into the implementation.

Paper: Learn How to Query from Unlabeled Data Streams in Federated Learning

Link: https://arxiv.org/html/2412.08138v2

Top comments (0)