When Dragons Go Missing: A Tale of URL Parameter Parsing in Phoenix LiveView

A debugging story about array parameters, pagination, and why choosing the right parsing function matters.

---

The Quest Begins

One day, Marci, an intrepid Elixir developer, was building an admin panel for a Monster Bestiary.

Everything was working smoothly until adventurers started reporting a bizarre bug: when they filtered by multiple monster types and tried to paginate through results, some of their selected filters mysteriously vanished.

Adventurers would select both "Dragons" and "Goblins" and when they clicked "Next Page" - poof! - suddenly only "Goblins" remained in their filters. The Dragons had vanished without a trace.

The Mysterious Disappearing Act

Here's what adventurers were experiencing:

1. Select "Dragons" and "Goblins" monster types
2. URL becomes: /admin/bestiary?monster_types[]=dragons&monster_types[]=goblins
3. Click "Next Page"
4. URL becomes: /admin/bestiary?monster_types[]=goblins&page=2
5. Only Goblin monsters shown on page 2

The "Dragons" filter was being dropped during pagination - completely unexpected behavior that left adventurers confused and frustrated.

Reproducing the Mystery

Marci knew the first step in any good debugging quest was to reproduce the problem reliably, so she wrote a test:

test "pagination preserves monster type filters when navigating between pages", %{
  conn: conn,
  bestiary: bestiary
} do
  # Create monsters for pagination testing
  create_monsters_for_pagination(bestiary, 10, :dragon)
  create_monsters_for_pagination(bestiary, 10, :goblin)
  create_monsters_for_pagination(bestiary, 5, :undead)

  # Start with Dragons and Goblins selected, per_page=5
  {:ok, view, _html} = live(conn, ~p"/admin/bestiary/#{bestiary}?per_page=5")

  # Filter to Dragons and Goblins only
  view
  |> element("#filter_form")
  |> render_change(%{filter: %{monster_types: ["dragons", "goblins"]}})

  # Should show 5 monsters (first page of Dragons + Goblins)
  assert_pagination_state(view, 5, "1-5 of 20")

  # Verify we have both Dragons and Goblins visible
  dragon_count = count_monsters_by_type(view, "Fire Drake")
  goblin_count = count_monsters_by_type(view, "Cave Goblin")
  assert dragon_count + goblin_count == 5

  # Test pagination through all pages
  for page <- 2..4 do
    # Go to next page
    view |> element("#pagination-next") |> render_click()

    # Should show next 5 monsters and maintain the same filters
    expected_range = "#{(page - 1) * 5 + 1}-#{min(page * 5, 20)} of 20"
    assert_pagination_state(view, 5, expected_range)

    # Verify we still have both Dragons and Goblins visible
    dragon_count = count_monsters_by_type(view, "Fire Drake")
    goblin_count = count_monsters_by_type(view, "Cave Goblin")
    assert dragon_count + goblin_count == 5

    # Verify the filter pills still show Dragons and Goblins as selected
    assert_monster_types_selected(view, ["dragons", "goblins"])
    assert_monster_types_not_selected(view, ["undead"])
  end
end

defp count_monsters_by_type(view, type_text) do
  render(view)
  |> Floki.find("[data-test-monster-log-block]")
  |> Enum.count(fn element ->
    Floki.text(element) =~ type_text
  end)
end

Marci's failing test captured what the adventurers were experiencing. The test would be her guide - as long as it failed, the mystery remained unsolved.

The Quest for Answers

Prepared with a reliable reproduction case, Marci began investigating. Her initial hunches led her down several false paths:

False Lead #1: Monster Filter Logic
Maybe the database query for filtering monsters was somehow broken? But testing showed that filtering worked as expected - it was only during pagination that monsters disappeared.

False Lead #2: Form Submission Issues
Perhaps the monster type selection form wasn't submitting array parameters correctly? But inspection of the form data showed it was submitting as intended: monster_types[]=dragons&monster_types[]=goblins.

False Lead #3: LiveView State Management
Could this be a complex state management issue where the LiveView was losing track of selected filters? But the state seemed fine - it was the URL parameters that were wrong.

The symptoms were particularly misleading:

• ✅ Single monster type filters worked
• ❌ Multiple monster type filters broke during pagination
• ✅ Pagination worked on pages without array filters

This made it seem like a complex interaction between the filtering system and pagination logic was causing the issue.

Tracking the Beast

Marci decided to trace what happens step-by-step when adventurers use pagination. The bug had to be somewhere in this flow:

When filters are working correctly:

• Adventurer selects dragons & goblins
• URL becomes: /admin/bestiary?monster_types[]=dragons&monster_types[]=goblins
• Database query runs and returns both dragons and goblins
• Page displays correctly with all monster types

Note: The URL doesn't show page=1&per_page=15 because those are default values that are omitted to keep the URLs clean.

When "Next Page" is clicked (where things go wrong):

• Adventurer clicks "Next Page" button
• Pagination component needs to build a new URL for page 2
  • Expected URL: /admin/bestiary?monster_types[]=dragons&monster_types[]=goblins&page=2&per_page=15
  • Actual URL: /admin/bestiary?monster_types[]=goblins&page=2&per_page=15
• Dragons disappear from the URL completely!

The pagination component was supposed to preserve existing filters while adding page parameters. Marci traced the problem to the merge_url_params/2 function, which was responsible for combining the current URL with new pagination parameters:

defp merge_url_params(url, params) do
  uri = URI.parse(url)
  query = URI.decode_query(uri.query || "")
  new_query = Map.merge(query, params)
  %{uri | query: URI.encode_query(new_query)}
end

When Marci added logs to this function with the URL parameters from the failing test, she discovered the cause of the problem:

# Input URL query: 
url_query = "monster_types[]=dragons&monster_types[]=goblins&page=1&per_page=15"

# What URI.decode_query/3 returns:
URI.decode_query(url_query)
# => %{"monster_types[]" => "goblins", "page" => "1", "per_page" => "15"}

The dragons were being dropped! URI.decode_query/3 only keeps the last value for repeated keys. When it encountered multiple monster_types[] parameters, it threw away everything except the final "goblins" value.

The Root Cause: Choosing the Wrong Spell

The issue lay in the fundamental difference between two similar-looking Elixir functions:

URI.decode_query/3 - General Purpose URL Parsing:

URI.decode_query("monster_types[]=dragons&monster_types[]=goblins")
# Returns: %{"monster_types[]" => "goblins"}
# ❌ Only keeps the last value, loses all others!

Plug.Conn.Query.decode/3 - HTML Form Array Parsing:

Plug.Conn.Query.decode("monster_types[]=dragons&monster_types[]=goblins")
# Returns: %{"monster_types[]" => ["dragons", "goblins"]}
# ✅ Correctly handles arrays, preserves all values!

URI.decode_query/3 is designed for general URL parsing where duplicate keys are unusual. Plug.Conn.Query.decode/4 is specifically designed for HTML form submissions, which commonly use array parameters like monster_types[]. It's the same function Phoenix uses internally when processing form data.

Taming the Beast

The solution was elegant - just two lines of code; using the right spell for the job:

# Before:
defp merge_url_params(url, params) do
  uri = URI.parse(url)
  query = URI.decode_query(uri.query || "") # ❌
  new_query = Map.merge(query, params)
  %{uri | query: URI.encode_query(new_query)} # ❌
end

# After:
defp merge_url_params(url, params) do
  uri = URI.parse(url)
  query = Plug.Conn.Query.decode(uri.query || "")  # ✅ Decodes arrays
  new_query = Map.merge(query, params)
  %{uri | query: Plug.Conn.Query.encode(new_query)}  # ✅ Encodes arrays
end

With this fix in place, Marci ran her test and success! With the test passing, and manual verification in the browser confirming that dragons were no longer disappearing when adventurers paginated through their bestiary, Marci was confident in her fix.

• ✅ All 4 pages of monsters displayed correctly
• ✅ Both Dragons and Goblins remained visible throughout pagination
• ✅ Filter pills stayed consistently selected
• ✅ The mystical pagination range appeared as expected

The Adventurer's Journal

Before declaring victory, Marci verified the fix wouldn't break existing functionality. The beauty of the solution was that Plug.Conn.Query.decode/4 is actually a superset of URI.decode_query/3 - it handles everything the old function did, plus array parameters.

This meant:

• All existing pagination with simple key-value parameters worked identically
• The fix was backward compatible
• The pagination component was now more robust for future array parameter use cases
• All other tests continued to pass

Back in her workshop, Marci reflected on the adventure. What had started as a complex filtering mystery turned out to be a simple case of using the wrong parsing function. The disappearing dragons had led her down paths involving database queries and state management, but the real culprit was hiding in plain sight - a two-line URL parsing function.

She realized that her reproduction test had been invaluable, not just for confirming the bug existed, but as a compass throughout the debugging journey. Every hypothesis she tested could be immediately validated or rejected by running that single test.

Most importantly, she'd discovered that in the Elixir ecosystem, seemingly similar functions can have subtle but crucial differences. URI.decode_query/3 and Plug.Conn.Query.decode/4 looked almost identical, but one was designed for general URLs while the other understood HTML form conventions. Since her data came from HTML forms with array parameters, she needed the form-aware version.

The fix was just two lines of code, but the debugging journey highlighted the importance of understanding the subtle differences between similar-looking functions and tracing problems back to their actual source rather than chasing symptoms.

Sometimes the dragons aren't really missing - they're just hiding in the URL parser.

---

Happy coding! 🎉