DEV Community: Roberts Guļāns

Live view: first tiny steps

Roberts Guļāns — Sun, 24 Mar 2019 20:17:50 +0000

A few days ago phoenix live view went public. I was anticipating its release since first sneak peek at elixir conf. Despite being in such early development stage, it already have great introduction, documentation and examples. There are also already many other community blogs on it. A great resource on what community is building already with it is #liveview tag on Twitter.

Assumptions

You know what live view is and have it set up.

Works on my machine

elixir: 1.8.1
phoenix: 1.4.1

We will create a world clock app, for user to select and see what time it is in what parts of the world.

Let's start by registering the live view directly in routes.

defmodule MyAppWeb.Router do
  ...

  scope "/", MyAppWeb do
    pipe_through :browser

    # Registering live view handler for specific URL
    live "/time", TimeLive
  end
end

One gotcha for registering live views in routes is to provide default layout. Otherwise, you might get a case when view HTML itself on the initial request will be returned, but with no surrounding HTML. Most importantly js includes that handles live view updates from frontend side will be missing.

defmodule MyAppWeb.Router do
  ...

  pipeline :browser do
    plug Phoenix.LiveView.Flash
    ...
    # Add default layout
    plug :put_layout, {MyAppWeb.LayoutView, :app}
  end

  ...
end

First tiny step, show UTC live time

defmodule MyAppWeb.TimeLive do
  @moduledoc """
  A module that renders and handles all interactions from ui
  """

  use Phoenix.LiveView

  @doc """
  Renders this simple html with ony one plain varaible output
  """
  def render(assigns) do
    ~L"<%= @time %>"
  end

  @doc """
  First initial view state setup
  - Registers tick every second
  - Delegates work to handle tick
  """
  def mount(_session, socket) do
    :timer.send_interval(1000, self(), :tick)

    {:ok, handle_tick(socket)}
  end

  @doc """
  Delegates work to handle tick
  """
  def handle_info(:tick, socket) do
    {:noreply, handle_tick(socket)}
  end

  @doc """
  Handles each tick.
  - Gets current time in a human readable format
  - saves it for view to use
  """
  defp handle_tick(socket) do
    time = NaiveDateTime.utc_now()
    assign(socket, :time, time)
  end
end

Workflow:

mount callback is called an initial state (current time) for the view is prepared;
render callback with previously prepared data is called and HTML rendered;
HTML is sent to the browser;
every second (each tick) handle_info callback with :tick message is called, which triggers view to recalculate current time;
live view detects state changes and triggers render callback again and new HTML is generated
HTML is sent to the browser (i assume there are some optimizations, like sending the only diff, or something like that. Will have to look into that).

Allow the user to choose timezone

I still don't quite get timezones in elixir. By default, elixir supports only Etc/UTC. Found out about tzdata. Setting it up is outside if this scope, but it was pretty easy.

defmodule MyAppWeb.TimeLive do
  ...

  @doc """
  Renders html via phoenix tempaltes
  """
  def render(assigns) do
    MyAppWeb.LiveView.render("time.html", assigns)
  end

  @doc """
  Now when mounting we assign all timezones (for the user to choose from dropdown)
  Assign changeset (using changeset, because will accept input from the user)
  """
  def mount(_session, socket) do
    :timer.send_interval(1000, self(), :tick)

    socket =
      socket
      |> assign(:timezones, Tzdata.zone_list())
      |> assign(:changeset, LocalTime.timezone_changeset(%{timezone: "Etc/UTC"}))
      |> handle_tick()

    {:ok, socket}
  end

  @doc """
  Accepting dropdown value change.
  Creates new changeset with newly selected timezone.
  """
  def handle_event("update_timezone", %{"local_time" => new_timezone}, socket) do
    socket =
      socket
      |> assign(:changeset, LocalTime.timezone_changeset(new_timezone))
      |> handle_tick()

    {:noreply, socket}
  end

  @doc """
  Now handling tick means to update time, not with `NaiveDateTime`, but for selected timezone. 
  """
  defp handle_tick(socket) do
    assign(socket, :changeset, LocalTime.update_time(socket.assigns.changeset))
  end
end

Extracted view template lib/my_app_web/templates/live/time.html.leex looks as follows:

<%= f = form_for @changeset, "#", [phx_change: "update_timezone"] %>
  <%= select f, :timezone, @timezones, promt: "Select timezone" %>
  <%= text_input f, :time, readonly: true %>
</form>

You probably noticed that there are multiple references to LocalTime module. for changesets to work properly I had to create an embedded schema. It will also be responsible for validating user input.

defmodule LocalTime do
  @moduledoc """
  Handling data integrty
  """
  use Ecto.Schema

  alias Ecto.Changeset

  embedded_schema do
    field :timezone, :string
    field :time, :string
  end

  @doc """
  Changeset that does heavy lifting of validating if user input is correct
  """
  def timezone_changeset(params \\ %{}) do
    %__MODULE__{}
    |> Changeset.cast(params, [:timezone])
    |> Changeset.validate_inclusion(:timezone, Tzdata.zone_list())
    |> Map.put(:action, :insert)
  end

  @doc """
  Updates time for the specified changeset.
  If a changeset is invalid, there is no point on updating time, as the user has specified the wrong timezone.
  If timezone is correct, create calculate current datetime for specified timezone 
  """
  def update_time(%Changeset{valid?: false} = changeset) do
    changeset
  end

  def update_time(%Changeset{changes: %{timezone: timezone}} = changeset) do
    {:ok, time} = DateTime.now(timezone)
    Changeset.force_change(changeset, :time, time)
  end
end

Conclusion

The live view seems great technology. There are still many things I need to grasp (probably more phoenix framework wise). Manipulating changesets seemed odd. LocalTime shouldn't update time via Changeset. Or maybe it should, and I'm just wanting to make things more complex than they should be. In either case, after heavily using vuejs for a year. This seems so similar, yet so different.

I wonder, will live view stack also grow, like all those modern js frameworks. Router for when you want to go to different page, but reload only part of the page. How will live view template nesting pan out? Some shared state manager, like redux or vuex. Will there eventually be a division of live view components in smart and stupid ones.

With all modern js craze, it seems that headless cmss, and crmss are the trend. The live view seems like a sane alternative when you don't want to maintain two stacks (backend, frontend) and make everything communicate via a json API. There is a place for that, don't get me wrong, but I already have seen many places, where it was introduced because for trendiness. And all complexity overhead that comes along with it was unnecessary.

For me, live view have this sweet spot, where I can see how I can do everything I was able to do with jquery (data related dom manipulation wise, don't know about animations jet), 80% of what I was able to do with vuejs, but with minimal new technology stack overhead (It will come down to component reusability, it such thing even is in roadmap).

What are your thoughts on it? Will or maybe already have used it? Did you like it? Do you see the place for it, in your toolbox? Let me know.

Have a great day.

P.S. If i werent clear on some points. If there is a bug in my code, or just things that i have skipped over, please let me know as well.

Whole code without comments

defmodule MyAppWeb.TimeLive do
  @moduledoc """
  A module that renders and handles all interactions from ui
  """

  use Phoenix.LiveView

  @doc """
  Renders this simple html with ony one plain varaible output
  """
  def render(assigns) do
    MyAppWeb.LiveView.render("time.html", assigns)
  end

  def mount(_session, socket) do
    :timer.send_interval(1000, self(), :tick)

    socket =
      socket
      |> assign(:timezones, Tzdata.zone_list())
      |> assign(:changeset, LocalTime.timezone_changeset(%{timezone: "Etc/UTC"}))
      |> handle_tick()

    {:ok, socket}
  end

  def handle_info(:tick, socket) do
    {:noreply, handle_tick(socket)}
  end

  def handle_event("update_timezone", %{"local_time" => new_timezone}, socket) do
    socket =
      socket
      |> assign(:changeset, LocalTime.timezone_changeset(new_timezone))
      |> handle_tick()

    {:noreply, socket}
  end

  defp handle_tick(socket) do
    assign(socket, :changeset, LocalTime.update_time(socket.assigns.changeset))
  end
end

defmodule LocalTime do
  use Ecto.Schema

  alias Ecto.Changeset

  embedded_schema do
    field :timezone, :string
    field :time, :string
  end

  def timezone_changeset(params \\ %{}) do
    %__MODULE__{}
    |> Changeset.cast(params, [:timezone])
    |> Changeset.validate_inclusion(:timezone, Tzdata.zone_list())
    |> Map.put(:action, :insert)
  end

  def update_time(%Changeset{valid?: false} = changeset) do
    changeset
  end

  def update_time(%Changeset{changes: %{timezone: timezone}} = changeset) do
    {:ok, time} = DateTime.now(timezone)
    Changeset.force_change(changeset, :time, time)
  end
end

#
<%= f = form_for @changeset, "#", [phx_change: "update_timezone"] %>
  <%= select f, :timezone, @timezones, promt: "Select timezone" %>
  <%= error_tag f, :timezone %>
  <%= text_input f, :time, readonly: true %>
</form>

Graphql journey. Part 3: pagination

Roberts Guļāns — Sun, 17 Mar 2019 18:41:35 +0000

Each system has to handle cases when there are zero data (404), has one instance (viewing this particular post), has few instances (tags assigned to posts) and many instances (dev.to posts. Cases when one can safely assume there will be too many to show in one place everything). This post is about the latter and most practical way to deal with datasets with unbounded size: pagination. But let's start with an easier user case.

Works on my machine

elixir: 1.8.1
phoenix: 1.4.2
absinthe: 1.4.16

Returning everything

defmodule MyAppWeb.Schema do
  use Absinthe.Schema

  @doc """
  Defining query posts which returns a list of posts from the repository
  """
  query do
    field :posts, list_of(:post) do
      resolve(&MyApp.PostsResolver.posts/3)
    end
  end

  @doc """
  Post structure
  """
  object :post do
    field :id, :integer
    field :title, :string
  end
end

defmodule MyApp.PostsRepository do
  @moduledoc """
  Represents the existing code boundary.
  Can be seen as internal code, that won't even know there is a graphql consumer.
  No graphql related code here
  """

  defmodule Post do
    @moduledoc """
    Simulates ecto schema
    """
    defstruct [:id, :title]
  end

  @doc """
  Simulates values coming from db (for example)
  """
  defp all_posts() do
    [
      %Post{id: 1, title: "1"},
      %Post{id: 2, title: "2"},
      %Post{id: 3, title: "3"},
      %Post{id: 4, title: "4"}
    ]
  end

  @doc """
  Returns all posts
  """
  def posts() do
    all_posts()
  end
end

defmodule MyApp.PostsResolver do
  @moduledoc """
  Resolves graphql query.
  Handles graphql call and maps it to necessary internal code.
  """

  @doc """
  Handles posts request and map it to internal posts repository to fetch results.
  """
  def posts(_, _, _) do
    {:ok, MyApp.PostsRepository.posts()}
  end
end

Building from previous posts in this dev.to series seems reasonable. Now we can execute the following graphql query and expect a result that is returned from MyApp.PostsRepository:

query {
  posts {
    id
    title
  }
}

Let's continue with paginating results and accessing further entries.

Pagination

defmodule MyAppWeb.Schema do
  ...

  query do
    field :posts, list_of(:post) do
      # Added required argument to provide page to fetch.
      arg(:page, non_null(:integer))
      resolve(&MyApp.PostsResolver.posts/3)
    end
  end
end

defmodule MyApp.PostsRepository do
  ...

  @doc """
  Returns posts for selected page
  """
  def posts(page, per_page) do
    offset = (page - 1) * per_page

    all_posts()
    |> Enum.drop(offset)
    |> Enum.take(per_page)
  end
end

defmodule MyApp.PostsResolver do
  @doc """
  How many posts show in a single page
  """
  @per_page 2

  @doc """
  Accept selected page and pass it to repository
  """
  def posts(_, %{page: page}, _) do
    {:ok, MyApp.PostsRepository.posts(page, @per_page)}
  end
end

Above you can see code changes to support pagination. It shows how small and trivial those changes actually are. Now we can provide page argument to fallowing graphql query and get the expected output.

query {
  posts(page: 3) {
    id
    title
  }
}

Most paginated endpoints have some meta information. For example how many pages in total, there are. Let's implement that as well.

Paginated with metadata

defmodule MyAppWeb.Schema do
  ...

  query do
    # Changed to return paginated posts object
    field :posts, :paginated_posts do
      arg(:page, non_null(:integer))
      resolve(&MyApp.PostsResolver.posts/3)
    end
  end

  @doc """
  Paginated posts object contains list of data and meta information
  """
  object :paginated_posts do
    field :results, list_of(:post)
    field :meta, :page_info
  end

  @doc """
  Meta information object structure
  """
  object :page_info do
    field :page, :integer
    field :total_pages, :integer
  end
end

defmodule MyApp.PostsRepository do
  ...

  @doc """
  How many posts there are
  """
  def count() do
    Enum.count(all_posts())
  end
end

defmodule MyApp.PostsResolver do
  ...

  @doc """
  Returns custom data set to support graphql defined structure
  """
  def posts(_, %{page: page}, _) do
    results = MyApp.PostsRepository.posts(page, @per_page)
    total_pages = Float.ceil(MyApp.PostsRepository.count() / @per_page)

    {:ok, %{results: results, meta: %{page: page, total_pages: total_pages}}}
  end
end

This is all needed changed code to implement simple pagination. And now graphql query looks like the following:

query {
  posts(page: 1) {
    results {
      id
      title
    },
    meta {
      page,
      totalPages
    }
  }
}

Conclusion

Pagination seemed easy for me to grasp. Building upon an understanding of previous posts, it felt natural.
Graphql methodology is to handle HTTP layer communication, and I like how unintrusive it is. They don't try to give some idiomatic pagination or something like that. Everything is a field, and you can resolve each field as you wish.
Each project could implement and require different structures to support all needs. Extended meta information for example. So take this example with a grain of salt, as not this is the "best" way to do it, nor it tries to be.

P.S. If you have any comments or I haven't been clear enough about some parts, please let me know :)

Uncommented final code version

defmodule MyAppWeb.Schema do
  use Absinthe.Schema

  query do
    field :posts, :paginated_posts do
      arg(:page, non_null(:integer))
      resolve(&MyApp.PostsResolver.posts/3)
    end
  end

  object :paginated_posts do
    field :results, list_of(:post)
    field :meta, :page_info
  end

  object :page_info do
    field :page, :integer
    field :total_pages, :integer
  end

  object :post do
    field :id, :integer
    field :title, :string
  end
end

defmodule MyApp.PostsRepository do
  defmodule Post do
    defstruct [:id, :title]
  end

  def all_posts() do
    [
      %Post{id: 1, title: "1"},
      %Post{id: 2, title: "2"},
      %Post{id: 3, title: "3"},
      %Post{id: 4, title: "4"}
    ]
  end

  def posts(page \\ 1, per_page) do
    offset = (page - 1) * per_page

    all_posts()
    |> Enum.drop(offset)
    |> Enum.take(per_page)
  end

  def count() do
    Enum.count(all_posts())
  end
end

defmodule MyApp.PostsResolver do
  @per_page 2

  def posts(_, %{page: page}, _) do
    results = MyApp.PostsRepository.posts(page, @per_page)
    total_pages = Float.ceil(MyApp.PostsRepository.count() / @per_page)

    {:ok, %{results: results, meta: %{page: page, total_pages: total_pages}}}
  end
end

Blazing fast CI suite for Elixir with Circle CI 2.1 for your libraries

Roberts Guļāns — Sun, 03 Mar 2019 21:13:57 +0000

There is a great post (stole title) about this topic already which helped me a lot to get started. But it has some shortcomings to make great CI for your library.

There is a great difference between client code (specific use case) and library (generic use case). One is used in one specific version (for example Elixir 1.8.1) while later one can be used in many versions (for example Elixir 1.8.1 for some new project and Elixir 1.3.4 for some brownfield project that never got updated).

How to make sure your library works on all supported versions? The best way would be to test in all supported environments. There are matrixes in Travis CI to tackle this particular use case, but they have made some quesionalbe decisions lately. Creating something like matrixes in Circle CI didn't seem as obvious or explicit as in Travis CI, but I made it work.

Asumptations

You know how to link Circle CI to your repo. I just authorized with my GitHub account and had to select my repo from the dropdown.
You know where to put Circle CI YAML configuration. (Psst... it's in .circleci/config.yml from repo root)

Testing library in the latest elixir version

version: 2

# Jobs are like specific executable setups
jobs:
  # Define new job with a name
  test-1.8.1:
    # How many instances of this particular job can be run in parallel
    parallelism: 1
    # Define which docker image use as the base
    docker:
      - image: elixir:1.8.1

    # What steps the current job should execute
    steps:
      # Go to the project directory
      - checkout

      # Elixir setup
      - run: mix local.hex --force
      - run: mix local.rebar --force

      # Library setup
      - run: MIX_ENV=test mix do deps.get --only test, deps.compile, compile

      # Run tests
      - run: mix test

# Workflows are like bird view of your CI.
# Define which jobs should be executed in what orders
workflows:
  version: 2
  # Define new workflow with a name
  testing_all_versions:
    # Define which jobs should be executed
    jobs:
      # Execute job named test-1.8.1
      - test-1.8.1

Please look into oficial documentation for extended descriptions and even more options.

Making it blazing fast

There is this neat feature of caching which can increase the execution of jobs greatly. Let's see it in action:

version: 2

jobs:
  test-1.8.1:
    parallelism: 1
    docker:
      - image: elixir:1.8.1

    steps:
      - checkout

      # Restoring state from previous executions
      - restore_cache:
          key: testing-{{ .Environment.CIRCLE_JOB }}

      - run: mix local.hex --force
      - run: mix local.rebar --force

      - run: MIX_ENV=test mix do deps.get --only test, deps.compile, compile

      # Save state after steps execution
      - save_cache:
          key: testing-{{ .Environment.CIRCLE_JOB }}
          paths:
            - _build
            - deps
            - ~/.mix

      - run: mix test

workflows:
  version: 2
  testing_all_versions:
    jobs:
      - test-1.8.1

There is no magic involved. restore_cache and save_cache are regular steps:

save_cache - saves all provided folder contents in the cache
restore_cache - puts cached folders in newly created docker image (each time a new job is executing)

All steps in between are executing as usual, but for example how much time does mix deps.get, mix deps.compile and mix compile takes on first execution and each consecutive execution? If there aren't cached folders each job execution will fetch dependencies, compile them and then compile the project from scratch. On the other hand, if those folders are cached, each consecutive time already will have downloaded and compiled dependencies. Only new dependencies will be downloaded and only changed code will be recompiled.

Testing library compatibility with multiple elixir versions. First try.

YAML has concept as anchors which is also used in post mentioned above.

My first try also was using this approach:

version: 2

jobs:
  # Added anchor with name test (&test)
  test-1.8.1: &test
    parallelism: 1
    docker:
      - image: elixir:1.8.1

    steps:
      - checkout

      - restore_cache:
          key: testing-{{ .Environment.CIRCLE_JOB }}

      - run: mix local.hex --force
      - run: mix local.rebar --force
      - run: MIX_ENV=test mix do deps.get --only test, deps.compile, compile

      - save_cache:
          key: testing-{{ .Environment.CIRCLE_JOB }}
          paths:
            - _build
            - deps
            - ~/.mix

      - run: mix test
  # another job with name
  test-1.7.4:
    # Import all steps from anchor named test
    <<: *test
    # Each property added overwrites imported properties from anchor
    docker:
      - image: elixir:1.7.4
  test-1.6.6:
    <<: *test
    docker:
      - image: elixir:1.6.6
  test-1.5.3:
    <<: *test
    docker:
      - image: elixir:1.5.3
  test-1.4.5:
    <<: *test
    docker:
      - image: elixir:1.4.5
  test-1.3.4:
    <<: *test
    docker:
      - image: elixir:1.3.4

workflows:
  version: 2
  testing_all_versions:
    jobs:
      - test-1.8.1
      - test-1.7.4
      - test-1.6.6
      - test-1.5.3
      - test-1.4.5
      - test-1.3.4

Basically, we reuse previously defined the job and rewrite the docker image on which to execute steps. Now the library is tested in all provided elixir versions.

While it works great for this specific use case, I would also like to test code quality against the latest elixir version with mix format --check-formatted (code is formatted correctly), MIX_ENV=test mix coveralls.circle (tests coverage), mix credo (code quality) and mix dialyzer (type checking). For each of those cases, I would need to change the last step (mix test) defined in an anchored job.

Anchors (as much as I know) don't support partial overwrites or appends, only full property overwrites. So for each case, where only the last step is different, I would have to rewrite all steps. Which still would be fast and done. But not for me. Duplication in such manner just gives future me more places to be cautious around, more error-prone and just takes more mental space.

Some time ago I decided to start blogging with some regularity. Finding things to write about isn't as easy as seemed. So mainly I try to write about things I just learned and hope that my findings will help not only others but future me as well.

Testing library compatibility with multiple elixir versions. Second try.

I decided to investigate it further and find a solution that I would be satisfied with. As of Circle CI 2.1, there is such thing as parameters. Let's use those instead of anchors:

version: 2.1

# Anchors are good for something tho
default_version: &default_version 1.8.1
default_steps: &default_steps
  - run: mix test

jobs:
  # Job with name build
  build:
    # Define allowed parameters
    parameters:
      # Parameter - version
      version:
        # Parameter deccription
        description: Elixir version
        # Parameter is expected to be a string
        type: string
        # Use default (latest) elixir version if none is provided
        default: *default_version
      # Parameter - execute
      execute:
        description: What steps to execute after build
        # Parameter is expected to be a list of executable steps
        type: steps
        # Use default steps if none are provided
        default: *default_steps

    parallelism: 1
    docker:
      # Image is provided dynamically, by concatenating 
      # - elixir: 
      # - the value provided via version parameter
      # 
      # Might look something like elixir:1.8.1, elixir:1.7.4
      - image: elixir:<< parameters.version >>

    steps:
      - checkout

      - restore_cache:
          # Cache key is provided dynamically by concatenating
          # - testing-elixir-v
          # - the value provided via version parameter
          #
          # Might look something like testing-elixir-v1.8.1, testing-elixir-v1.7.4
          key: testing-elixir-v<< parameters.version >>

      - run: mix local.hex --force
      - run: mix local.rebar --force
      - run: MIX_ENV=test mix do deps.get --only test, deps.compile, compile

      - save_cache:
          # Cache key is provided dynamically by concatenating
          # - testing-elixir-v
          # - the value provided via version parameter
          #
          # Might look something like testing-elixir-v1.8.1, testing-elixir-v1.7.4
          key: testing-elixir-v<< parameters.version >>
          paths:
            - _build
            - deps
            - ~/.mix

      # Execute steps provided via execute parameter.
      - steps: << parameters.execute >>

workflows:
  version: 2.1
  testing_all_versions:
    jobs:
      # Job to execute (the only job actually provided)
      - build:
          # Give it a custom name
          # Otherwise all jobs would have build-1, build-2, ... names
          name: "Test in elixir 1.8.1"
          # Ignore execute parameter. Will use default steps (mix test)
          # Provide version (defined in job parameters)
          # Used in
          # - defining docker image (which elixir version to use)
          # - as a cache key, so each version would have a separate cache
          #   all job calls with the same version will share cache files
          version: 1.8.1
      - build:
          # Provide a different name
          name: "Test in elixir 1.7.4"
          # Provide different elixir version
          version: 1.7.4
      - build:
          name: "Test in elixir 1.6.6"
          version: 1.6.6
      - build:
          name: "Test in elixir 1.5.3"
          version: 1.5.3
      - build:
          name: "Test in elixir 1.4.5"
          version: 1.4.5
      - build:
          name: "Test in elixir 1.3.4"
          version: 1.3.4

As you can see, now we have one boilerplate job, that can be called with different parameters (something like a function). So we only need to provide the elixir version and actual meaningful steps if needed. In this case, we can only provide a version parameter as a default step to execute are mix test one we would have provided.

Testing library compatibility with multiple elixir versions and code quality

Now we are ready to implement additional tests, that are related to code quality: mix format --check-formatted (code is formatted correctly), MIX_ENV=test mix coveralls.circle (tests coverage), mix credo (code quality) and mix dialyzer (type checking).

version: 2.1

default_version: &default_version 1.8.1
default_steps: &default_steps
  - run: mix test

jobs:
  build:
    parameters:
      version:
        description: Elixir version
        type: string
        default: *default_version
      execute:
        description: What steps to execute after build
        type: steps
        default: *default_steps

    parallelism: 1
    docker:
      - image: elixir:<< parameters.version >>

    steps:
      - checkout

      - restore_cache:
          key: testing-elixir-v<< parameters.version >>

      - run: mix local.hex --force
      - run: mix local.rebar --force
      - run: MIX_ENV=test mix do deps.get --only test, deps.compile, compile

      - save_cache:
          key: testing-elixir-v<< parameters.version >>
          paths:
            - _build
            - deps
            - ~/.mix

      - steps: << parameters.execute >>

workflows:
  version: 2.1
  testing_all_versions:
    jobs:
      - build:
          name: "Test in elixir 1.8.1"
          version: 1.8.1
      - build:
          name: "Test in elixir 1.7.4"
          version: 1.7.4
      - build:
          name: "Test in elixir 1.6.6"
          version: 1.6.6
      - build:
          name: "Test in elixir 1.5.3"
          version: 1.5.3
      - build:
          name: "Test in elixir 1.4.5"
          version: 1.4.5
      - build:
          name: "Test in elixir 1.3.4"
          version: 1.3.4
  # Add aditional workflow
  validate_code_quality:
    jobs:
      # Call build job
      - build:
          # Provide custom name
          name: "Code formatted correclty"
          # Do not provide version parameter (use default one)
          # Provide execute parameter with custom steps
          execute:
            # This step will be executed instead of default one
            - run: mix format --check-formatted
      - build:
          name: "Code style check"
          execute:
            - run: mix credo
      - build:
          name: "Type check"
          execute:
            - run: mix dialyzer
      - build:
          name: "Tests coverge"
          execute:
            - run:
                command: MIX_ENV=test mix coveralls.circle
                environment:
                  COVERALLS_REPO_TOKEN: 5wI8JIzygEDIF4A03KNIKWdOmVK2A8dMC

That's it. Now Circle CI will both test library in all specified elixir versions and checks for code quality (with formater, credo, dialyzer, and coveralls).

Circle CI CLI tool

While this example can be reused in many cases, often you might want to implement custom configuration. Committing and publishing each time to see Circle CI fail because of misconfiguration is redundant and time-consuming. But no worries, Circle CI have created a cli tool.

While it has few commands, one most useful seems to be circleci config validate which checks if the configuration file is written correctly. When you commit and push you will already know it will work as expected. No need for 20 commits, that all say fixed circle ci config (as I did in beginning).

Conclusion

Circle CI 2.1 brings many improvements. One of which is parameters. There are many different ways how to reuse config parts.
Circle CI is fast, it rarely takes more than a minute from the time I push code, to time all jobs have been executed. Probably has something to do with a small library having few tests to cover everything. I also don't use credo and dialyzer in my current setup. Who knows maybe they like to execute much longer.
Circle CI support is also fast. They responded within 24h (in the weekend).
Circle CI CLI is a great companion tool to help you configure your CI environment with a faster feedback loop.
It was interesting to try something outside my usual realm. While I hade a few bums here and there, it was a great experience. Setup was straight forward and set up the next projects will be even easier.

P.S. If you have any questions or ideas on how to improve Circle CI config or blog post itself, please let me know :)

GraphQl with absinthe: How to map data to different structures

Roberts Guļāns — Sun, 24 Feb 2019 16:57:57 +0000

After starting using graphql with absinthe and discovering some gotchas everything went nice and smooth. After a few days tho, I encountered a case that our data source has a different data structure than one that has to be defined in graphql. While it was mostly straight forward to implement, I didn't find many resources on how to do it.

Assertions

you are familiar with elixir. Where to learn?
you are familiar with both graphql and absinthe.

Works on my machine

elixir: 1.7.3
phoenix: 1.4.0
absinthe: 1.4.13

Bussiness rules

Existing user entity has field name, which oddly enough doesn't contain a string, but a map (saved as json in DB) with two keys (first and last). But graphql has to have first_name, second_name and their combined value as full_name to be alongside id.

Initial setup

To start to implement needed business requirements, as usual, there is some bootstrap code that is needed. This is no exception.

We will have to implement four modules:

MyApp.Schema -> graphql structures, queries and mutations.
MyApp.UsersRepository -> represents existing repository.
MyApp.UsersRepository.User -> represents exisitng ecto schema.
MyApp.UsersResolver -> graphql resolver callbacks.

Both MyApp.UsersRepository and MyApp.UsersRepository.User in this example is outside of the graphql boundary. This part of code should live in a different context, and shouldn't be influenced in any way by the fact that now it has a new consumer (graphql) that requires different data structure.

Note that MyApp.Schema defines an object user that has fields described in the business rules, but MyApp.UsersRepository expect name values to be nested under name key (representing existing repository structure).

defmodule MyApp.Schema do
  @moduledoc """
    GraphQL schema description (available objects, queries and mutations)
  """

  use Absinthe.Schema

  alias MyApp.UsersResolver

  @doc """
    User object with few scalar fields
  """
  object :user do
    field :id, :integer
    field :first_name, :string
    field :last_name, :string
    field :full_name, :string
  end
end

defmodule MyApp.UsersRepository do
  @moduledoc """
    Actual repository, used as change boundary.
    This code will not change at all during this blog post.
  """

  defmodule User do
    @moduledoc """
      In a real-life scenario, this most likely would be an ecto schema.

      For sake of example, simple struct will also do.
    """
    defstruct id: nil, name: %{"first" => nil, "last" => nil}
  end

  @doc """
    Gets a list of users from the data source.
    In real life scenario most likely from DB via ecto.

    In this case list with one hardcoded user.
  """
  def get_all() do
    [%User{id: 1, name: %{"first" => "John", "last" => "Doe"}}]
  end

  @doc """
    Creates a user in a data store.
    In real life scenario most likely to DB via ecto.

    In this case, checks if data has required structure and if so, add id (like autogenerated id value).
  """
  def create(%User{name: %{"first" => _, "last" => _}} = user), do: {:ok, %{user | id: 10}}
  def create(_), do: {:error, "Wrong structure"}
end

defmodule MyApp.UsersResolver do
  @moduledoc """
    Handles data processing for graphql.

    Sort of like repository, but defined as graphql resolve callbacks.
  """
  alias MyApp.UsersRepository
  alias MyApp.UsersRepository.User
end

Receiving data

We need to be able to make the following query:

query {
  users {
    firstName,
    lastName,
    fullName
  }
}

So we need to add query block with users field.
In MyApp.UsersResolver we make a function that is defined as a resolver for users query. The only difference from the usual case where data maps 1:1 is that we use &Enum.map/2 to iterate over results from &MyApp.UsersRepository.get_all/0 with &to_graphql/1 and can make any structure we need.

Note that graphql by default expects atom keys. It can be changed if needed.

defmodule MyApp.Schema do
  ...

  @doc """
    Makes query `users` available for API consumers. 
    The query is expected to return a list of `user` objects.
    `&UsersResolver.fetch_users/3` is responsible to make that happen
  """
  query do
    field :users, list_of(:user), resolve: &UsersResolver.fetch_users/3
  end
end

defmodule MyApp.UsersResolver do
  ...

  @doc """
    Fetches all users from the repository.
  """
  def fetch_users(_, _, _) do
    users = UsersRepository.get_all() |> Enum.map(&to_graphql/1)
    {:ok, users}
  end

  @doc """
    Map data from user instance to one required by graphql api.
  """
  defp to_graphql(%User{id: id, name: %{"first" => first, "last" => last}}) do
    %{id: id, first_name: first, last_name: last, full_name: "#{first} #{last}"}
  end
end

now executing the required query we get expected output:

{
  "data": {
    "users": [
      {
        "lastName": "Doe",
        "fullName": "John Doe",
        "firstName": "John"
      }
    ]
  }
}

Mutation mapping

This is a mutation we are required to be able to execute:

mutation {
  createUser(firstName: "Doe", lastName: "John") {
    id
    fullName
  }
}

Here things get a little bit more involved in absinthe. Not much, but still.
First of all, now we crate mutation. Then in create_user resolver, we take the second argument, as it is where arguments passed through graphql are located.
We do a similar map as previously with to_graphql but in reverse.

defmodule MyApp.Schema do
  ...

  @doc """
    Mutation to create a new user
    Two fields are required, and mutation is expected to return created user instance.
  """
  mutation do
    field :create_user, :user do
      arg :first_name, non_null(:string)
      arg :last_name, non_null(:string)

      resolve &UsersResolver.create_user/3
    end
  end
end

defmodule MyApp.UsersResolver do
  ...

  @doc """
    Creates a new user to repository.
  """
  def create_user(_, args, _) do
    args |> from_graphql |> UsersRepository.create()
  end

  @doc """
    Converts graphql structure to one required by the repository.
  """
  defp from_graphql(%{first_name: first, last_name: last}) do
    %User{name: %{"first" => first, "last" => last}}
  end
end

And we expect to get as a response:

{
  "data": {
    "createUser": {
      "id": 10,
      "fullName": "Doe John"
    }
  }
}

but that isn't what we actually get. fullName is null. Reason for it is that &MyApp.UsersResolver. create_user/3 resolver callback returns User struct (received from MyApp.UsersRepository.create/1). The resolver should always return graphql valid structure. To fix it, we need to map User struct value back to graphql with previously used &to_graphql/1 like so:

defmodule MyApp.UsersResolver do
  ...

  def create_user(_, args, _) do
    args 
    |> from_graphql 
    |> UsersRepository.create()
    |> case do
      {:ok, user} -> {:ok, to_graphql(user)}
      error -> error
    end
  end
end

Now required mutation actually returns what was expected previously.

In conclusion

graphql doesn't interfere with data resolving. Resolve callbacks are plain elixir code, you can manipulate data as ever you please. Maps expect to have atom keys by default, but it can be changed.
Creating MyApp.UsersResolver in between MyApp.Schema and MyApp.UsersRepository is a great way to split models into sperate use cases. Each model deals only with one specific task.
resolvers for mutations, that return object, should take in consideration, that mutation callback has to return object in graphql valid structure (mutate return data if needed).
mutation callback is the only place that does two jobs, saving data and preparing a response. If we applied something CQSish that mutation at most would return the newly generated id, so no response mapping would be needed in that case.

I'm still pretty new to graphql, but I really like what I see.

P.S. If you have any feedback, suggestion, question or thoughts about this topic, please let me know :)

P.P.S Whole code without comments

defmodule MyApp.Schema do
  use Absinthe.Schema

  alias MyApp.UsersResolver

  object :user do
    field :id, :integer
    field :first_name, :string
    field :last_name, :string
    field :full_name, :string
  end

  query do
    field :users, list_of(:user), resolve: &UsersResolver.fetch_users/3
  end

  mutation do
    field :create_user, :user do
      arg :first_name, non_null(:string)
      arg :last_name, non_null(:string)

      resolve &UsersResolver.create_user/3
    end
  end
end

defmodule MyApp.UsersRepository do
  defmodule User do
    defstruct id: nil, name: %{"first" => nil, "last" => nil}
  end

  def get_all() do
    [%User{id: 1, name: %{"first" => "John", "last" => "Doe"}}]
  end

  def create(%User{name: %{"first" => _, "last" => _}} = user), do: {:ok, %{user | id: 10}}
  def create(_), do: {:error, "Wrong structure"}
end

defmodule MyApp.UsersResolver do
  alias MyApp.UsersRepository
  alias MyApp.UsersRepository.User

  def fetch_users(_, _, _) do
    users = UsersRepository.get_all() |> Enum.map(&to_graphql/1)
    {:ok, users}
  end

  def create_user(_, args, _) do
    args
    |> from_graphql
    |> UsersRepository.create()
    |> case do
      {:ok, user} -> {:ok, to_graphql(user)}
      error -> error
    end
  end

  defp to_graphql(%User{id: id, name: %{"first" => first, "last" => last}}) do
    %{id: id, first_name: first, last_name: last, full_name: "#{first} #{last}"}
  end

  defp from_graphql(%{first_name: first, last_name: last}) do
    %User{name: %{"first" => first, "last" => last}}
  end
end

GraphQl with absinthe

Roberts Guļāns — Thu, 14 Feb 2019 20:15:56 +0000

There are a ton of graphql tutorials already. This is more like solidifying my understanding of the topic. If you want to learn it, there probably is better resources to follow.

Gotcha 1: Everything is a field.

Literally, everything is a "field". What do I mean?

Each field can be a predefined scalar value (string boolean, float, id or integer) or custom scalar value

defmodule MyApp.Schema do
  use Absinthe.Schema

  # All fields in this scope can be publicly accessible
  query do
    # Defining field `name` with type string
    field :name, :string
  end
end

Now we can execute query:

query {
  name
}

and expect the following output:

{
  "data": {
    "name": null
  }
}

There's nothing to it because we haven't told absynthe how to resolve this field. Let's for now tell it to return hardcoded value.

defmodule MyApp.Schema do
  use Absinthe.Schema

  # All fields in the query scope can be publicly accessible
  query do
    # Defining field `name` with type string and resolver for it
    field :name, :string, resolve: fn _, _, _ -> {:ok, "John Doe"} end
  end
end

Now executing the same query we get John Doe as value for the name field instead of null

{
  "data": {
    "name": "John Doe"
  }
}

The resolver callback cannot return an object if the type is provided as a string.

Gotcha 2: Fields can also be complex structures (list, map)

Often field will contain more complex value than a scalar. Especially if the field is at root level:

defmodule MyApp.Schema do
  use Absinthe.Schema

  # All fields in the query scope can be publicly accessible
  query do
    # Defining field `user` with type `user`
    field :user, :user
  end

  # Custom defined object `user` that has one field `name` and resolver for it
  object :user do
    field :name, :string, resolve: fn _, _, _ -> {:ok, "John Doe"} end
  end
end

Now we can execute query:

query {
  user {
    name
  }
}

and expect the following output:

{
  "data": {
    "user": {
      "name": "John Doe"
    }
  }
}

but actually, we get

{
  "data": {
    "user": {
      "name": null
    }
  }
}

Hmmm... so... I can not resolve deeply nested structures. I need to resolve at the root level. At least it seems so.

defmodule MyApp.Schema do
  use Absinthe.Schema

  # All fields in the query scope can be publicly accessible 
  query do
    # Defining field `user` with type `user` and resolver for it
    field :user, :user, resolve: fn _, _, _ -> {:ok, %{name: "John Doe"}} end
  end

  # Custom defined object `user` that has one field `name`
  object :user do
    field :name, :string
  end
end

While executing the previous query we now indeed get the expected output:

{
  "data": {
    "user": {
      "name": "John Doe"
    }
  }
}

If the parent level doesn't provide field, the child field resolver callback isn't called. So we could get the same effect by combining resolvers from both levels:

defmodule MyApp.Schema do
  use Absinthe.Schema

  # All fields in the query scope can be publicly accessible
  query do
    # Defining field `user` with type `user` and resolver for it
    field :user, :user, resolve: fn _, _, _ -> {:ok, %{name: nil}} end
  end


  # Custom defined object `user` that has one field `name` and resolver for it
  object :user do
    field :name, :string, resolve: fn _, _, _ -> {:ok, "John Doe"} end
  end
 end

We can even combine resolvers from both levels and expect the same output:

defmodule MyApp.Schema do
  use Absinthe.Schema

  # All fields in the query scope can be publicly accessible
  query do
    # Defining field `user` with type `user` and resolver for it
    field :user, :user, resolve: fn _, _, _ -> {:ok, %{name: "John"}} end
  end

  # Custom defined object `user` that has one field `name` and resolver for it
  object :user do
    field :name, :string, resolve: fn %{name: name}, _, _ -> {:ok, name <> " Doe"} end
  end
end

Gotcha 2.1: Fields can also be complex structures

All tutorials I were looking into, were tightly coupled with ecto, so it somehow slipped my attention, how exactly field structures were working. Here you could see the even deeper nested structure. Each user have permissions map field with two values is_admin? and role.

defmodule MyApp.Schema do
  use Absinthe.Schema

  # All fields in the query scope can be publicly accessible
  query do
    # Defining field `user` with type `user` and resolver for it
    field :user, :user, resolve: &resolve_user/3
  end

  # Custom defined object `user` that has two fields `name` and `permissions` and resolver for the latter
  object :user do
    field :name, :string
    field :permissions, :permissions, resolve: &resolve_permisisons/3
  end

  # Custom defined object `permissions` that has two fields
  object :permissions do
    field :is_admin, :boolean
    field :role, :string
  end

  defp resolve_user(_, _, _) do
    # Here as well could happen DB query, or reading data from any imaginable data source possible
    # For now simulating that JSON value is retrieved from a data source in permissions field
    user = %{
      name: "John Doe",
      permissions: Jason.encode!(%{is_admin: true, role: "admin"})
    }

    {:ok, user}
  end

  defp resolve_permisisons(%{permissions: permissions}, _, _) do
    Jason.decode(permissions, keys: :atoms!)
  end
end

Needless to say that such hierarchies could go as deep as necessary. There are some security concerns regarding depth.

It is possible to retrieve data from one data source, and while resolving its children, mutate its values (in this case decoding JSON), but even reach in other data source for extra data. Be careful tho, 1+n can creep very fast.

In conclusion

From the graphql perspective, there isn't some special root elements, root objects or something like that. Everything is a field, and you have complete control over what (string, integer, some custom scalar value or object) each field represents and how data for a specific field is retrieved and processed.

As with erlang/elixir itself, such simple concept as processes with mailboxes have evolved into such stable environments, the graphql went similar path, by choosing to implement as little new concepts as possible. When right building blocks are refined, solutions on those blocks can be built not only faster, but also more stable.

I sort of understand hype train around graphql. Will see how my journey through mutations, paginations, and other seemingly more complex use cases will go, but that's a story for another day.

P.S. Feedback is welcome