DEV Community: Marcøs

Elixir - csv import

Marcøs — Thu, 29 Oct 2020 03:08:08 +0000

See Files on Github
country-codes.csv
create_country_table.exs

This post expands on this article from 'Experimenting with Code' to only import a subset of csv columns into Postgres using Ecto.

In this case, we are only concerned with the code and name columns from country-codes.csv. This example uses File.stream() and CSV.decode!() to get just the two columns and pass it to the COPY command.

file_path
|> File.stream!()
|> CSV.decode!(headers: true)
|> Stream.map(fn row ->
  [
    row["code"],
    row["name"]
  ]
end)
|> CSV.encode()
|> Enum.into(stream)

Elixir - schema drop

Marcøs — Fri, 07 Aug 2020 19:43:49 +0000

See Files on Github
config.exs
mix.exs
schema_drop.exs

There may be cases when you share a database (but not a schema) with other applications. So using mix ecto.drop may not be an option. In this case, dropping the schema may be a better alternative.

First configure your ecto migrations settings in config.exs.

config :portishead, Portishead.Repo,
  migration_default_prefix: "portishead",
  migration_source: "portishead_schema_migrations"

Add schema_drop.exs script to the priv/repo folder. The script gets the schema and migration table from the config and runs raw sql to drop them. This script is using PL/pgSQL.

Repo.query!("DROP SCHEMA IF EXISTS #{schema} CASCADE")
Repo.query!("DROP TABLE IF EXISTS #{migration_table}")

In mix.exs, add an alias to drop the schema

  defp aliases do
    [
      "ecto.schema.drop": ["run priv/repo/schema_drop.exs"]
    ]
  end

From the shell, you can now run

$ mix ecto.schema.drop

or you can add an aliases to use ecto.schema.drop

  defp aliases do
    [
      "ecto.schema.reset": ["ecto.schema.drop", "ecto.setup"]
    ]
  end

Elixir - be sorted, not sordid

Marcøs — Mon, 15 Jun 2020 20:47:50 +0000

See code on GitHub

I notice that when devs use libraries like ExMachina to generate fake data for tests, they forget to test for case insensitivity. So they end up with unexpected results like all the uppercase letters appearing before all of the lowercase letters.

iex(1)> Enum.sort(["b", "a", "c", "B", "A", "C"])
["A", "B", "C", "a", "b", "c"]

To help with testing, I've written a helper function to randomize the case of a string.

  def random_case(string) when is_binary(string) do

It can be combined with the [sequence] function (https://hexdocs.pm/ex_machina/ExMachina.html#sequence/1) in a factory

def article_factory do
  %Article{
    title: sequence("Article Title") |> ExMachinaHelper.random_case()
  }
end

There is some overhead in changing the case of strings so only choose the fields related to sorting.

Elixir setup - being consistent with schema

Marcøs — Fri, 12 Jun 2020 02:53:37 +0000

See Files on Github
common.ex
band.ex

It's typical to see timestamps() in every schema file, but what if you want to have the same primary key, schema prefix, or subset of columns in all the schema files.

First create your own using macro to handle the use/import statements along with any schema attributes.

  defmacro __using__(_) do
    quote do
      use Ecto.Schema
      import Ecto.Changeset
      import Portishead.Schema.Common

      @primary_key {:uuid, :binary_id, autogenerate: true}
      @schema_prefix "premarcos"
    end
  end

Then define a macro that contains the common fields across all your schemas. You can use timestamps() or define your own date fields instead like created_at.

  defmacro common_fields do
    quote do
      # field :created_at, :utc_datetime_usec, autogenerate: {Ecto.Schema, DateTime.utc_now(), []}
      field :metadata, :string
      timestamps()
    end
  end

Finally, in your schema file, use the module that was just created and call the common_fields macro.

defmodule Portishead.Schema.Band do
  use Portishead.Schema.Common

  schema "band" do
    common_fields()
  end

Also worth reading

https://bernheisel.com/blog/ecto_changeset_tips/

Elixir project setup - config.exs

Marcøs — Tue, 28 Jan 2020 04:27:55 +0000

I'll be writing about typical changes I make to the default phoenix project

See file on GitHub

NOTE: This article assumes you will using Ecto, and the database tables use UUID as the primary key.

If you plan on running mix phoenix.gen.model to generate Ecto models, set the generators key to use binary_id for schema’s primary key.

config :portishead,
  generators: [binary_id: true]

For mix ecto.migrate with create table syntax, set the migration_primary_key to use uuid. migration_default_prefix is the schema name of the table in postgresql.

config :portishead, Portishead.Repo,
  migration_primary_key: [name: :uuid, type: :uuid],
  migration_default_prefix: "portishead",
  migration_source: "portishead_schema_migrations"

Elixir project setup - mix.exs

Marcøs — Tue, 28 Jan 2020 04:27:43 +0000

I'll be writing about typical changes I make to the default phoenix

See file on GitHub

Update the minimum elixir version allowed. If you're working with other devs, this will force them to keep their local environment up to date.

      elixir: ">= 1.9.0",

Override the default tasks for compile and ecto.migrate.

  defp aliases do
    [
      compile: ["compile --warnings-as-errors"],
      "ecto.migrate": ["ecto.migrate --log-sql"],
    ...
    ]

The compile alias is useful make sure that the warnings-as-errors flag is always enabled, even when running mix test or mix release. The ecto.migrate flag is to display the raw sql for migration scripts, not just the Ecto Query DSL.