Elixir - Simple Req Cookie Jar

Carlo Colombo — Tue, 13 Feb 2024 08:24:49 +0000

A simple req cookie jar plugin. It can be used when automating or scraping websites that implement cookie based sessions. The cookies are set using set-cookie header and are sent back to the server in the headers of each following request. The implementation is naive and probably misses advanced and corner cases.

# create the cookie jar
CookieJar.new()

# attach the plugin to the req
req = Req.new(base_url: "http://baseurl.example.com")
|> CookieJar.attach()

# when the response includes the set-cookie header, it store its value in the jar
%{body: "Ok."} = req
|> Req.post!(
         url: "/api/v2/auth/login",
         form: [username: "admin", password: "password"])

# all subsequent requests have the cookie set in the headers
%{status: 200} = req
|> Req.get!(url: "/api/v2/info?filter=foo")

The cookie is stored in an Agent. Req response and request steps are respectively used to collect the cookie string and to add to the cookie header when requests are performed.

defmodule ReqCookieJar do
  use Agent

  def new() do
    Agent.start_link(fn -> "" end, name: __MODULE__)
  end

  defp get_cookie do
    Agent.get(__MODULE__, & &1)
  end

  defp set_cookie([]), do: nil
  defp set_cookie(val) do
    Agent.update(__MODULE__, fn (_) -> val end)
  end

  def attach(%Req.Request{} = request) do
    request
    |> Req.Request.append_response_steps(
      cookie_jar: fn({req, res})->
        Req.Response.get_header(res, "set-cookie")
        |> set_cookie()

        {req,res}
      end
    )
    |> Req.Request.append_request_steps(
      cookie_jar: &Req.Request.put_header(&1, "cookie", get_cookie())
    )
  end
end

jq fzf powered filtering

Carlo Colombo — Tue, 12 Dec 2023 18:00:00 +0000

When needing to explore a JSON file for a long time I just reached into my history and looked for a oneliner that I copied from a github gist echo '' | fzf --print-query --preview "cat *.json | jq {q}". This worked quite well, but I have now decided to improve it leveraging the fzf features.

Features

Supports reading from stdin (pipe or -) and path
Keeps history of selected filters and presenting them as options for fzf.
Prints the result of applying the filter the input

The script

function jqz () {
    if [[ -z $1 ]] || [[ $1 == "-" ]]; then
        input=$(mktemp)
        trap "rm -f '$input'" EXIT
        cat /dev/stdin > "$input"
    else
        input="$1"
    fi

    local q=$(<$HOME/.jqz-history  \
    | fzf \
        --history=$HOME/.jqz-history \
        --query '.' \
        --prompt 'jq filter> ' \
        --bind "tab:replace-query"\
        --bind "return:print-query" \
        --bind "alt-up:preview-page-up,alt-down:preview-page-down" \
        --bind 'focus:transform-preview-label:echo [ {} ]' \
        --header 'TAB: select filter' \
        --preview "<$input | jq --color-output -r  {q}")

    jq -r "$q" "$input"
}

fzf options

--history saves the selected filters to this file, it enable going to the previous selection with CTRL-N and CTRL-P. The history file is also used to provide options to pick from.
--prompt shows a customized prompt
--bind binds an event or a keybinding to an action, check fzf documentation for complete list. Here is used to scroll the preview, to update the preview title, and to select a filter.
--header text that is presented above the query
--query the default query

Supporting both stdin and paths

I got this part from Implementing a jq REPL with fzf, it allows to pipe in json {"a":42} | jqz or to read from a file jqz /tmp/some-file.json