DEV Community: Rogério Ramos

Using Duration with Ecto+Postgrex

Rogério Ramos — Sun, 23 Mar 2025 15:40:56 +0000

Duration: a useful time interval representation

Duration is covered by ISO 8601 standard that deal with time data and defines the amount of time in a given interval

One might think in the simpler and classic example to explain a duration: the difference between two dates or time

What is the age of a person?
How many days to the due date of that expensive loan?
The countdown in days or hours to the most anticipated concert

Thus a data type to represent a duration of a given time interval becomes useful in many use cases

The readable representation for human beings of a duration is, according to the ISO 8601 standard, a letter representing the unit of time followed by a number representing the amount of this unit. So here we go:

P is a marker used to split the Período portion of a given duration, represented as such:

Y	represents the number of years	(Years)
M	represents the number of months	(Months)
W	represents the number of weeks	(Weeks)
D	represents the number of days	(Days)

T is a marker used to split the Time portion of a given duration, represented as such:

H	represents the number of hours	(Hours)
M	represents the number of minutes	(Minutes)
S	represents the number of seconds	(Seconds)

The P3W2DT10H45M represents a duration of 3 weeks, 2 days, 10 hours e 45 minutes

Ecto, Postgrex and Duration

The Elixir structure %Duration{} and its functions were introduced in the version 1.17 and in previous versions, according to the documentation, the used structure to decode time intervals on PostgreSQL was Postgrex.Interval that so far is still the default structure

In the following snippets on can see the schema and migration definitions for a fictitious set of data as well as the materialization on database

Schema

schema "records" do
  field :name, :string
  field :description, :string 
  field :start, :utc_datetime
  field :duration, :duration
end

Migration

def change do
   create table(:records, primary_key: false) do
     add :id, :uuid, primary_key: true
     add :name, :string
     add :description, :string
     add :start, :utc_datetime
     add :duration, :duration

     timestamps(type: :utc_datetime)
   end
 end

Table Structure

   Column    |              Type              | Collation | Nullable | Default
-------------+--------------------------------+-----------+----------+---------+
 id          | uuid                           |           | not null |         |
 name        | character varying(255)         |           |          |         |
 description | character varying(255)         |           |          |         |
 start       | timestamp(0) without time zone |           |          |         |
 duration    | interval                       |           |          |         |

Persisted Data

                  id                  |    name    |     description      | start | duration |
--------------------------------------+------------+----------------------+-------+----------+
 664e5fd6-17ab-4a27-8799-9ebca4c603e5 | Record AA | The Record AA descr   |       | 11 days  |

What went wrong and why I brought this topic?

As the default structure mapped to PostgreSQL's intervals is Postgrex.Interval when the :duration type is defined in the schema and also in the migrations the value is correctly persisted in the other hand when it's retrieved the interval cannot be mapped to %Duration{} and the following error is thrown:

(ArgumentError) cannot load %Postgrex.Interval{months: 0, days: 11, secs: 0, microsecs: 0} as type :duration for field :duration

When someone is working in a product is desired to keep the focus in the work being done and mainly in the business domain, sometimes mixed up infrastructure code or external libraries might cause technical debts, maintenance and perhaps the lack of code readability and clearness

It's important to ensure a good level of homogenized code, leverage the usage of standard libraries. In the case of Duration structure it was supported but due my lack of knowledge it was not working properly.

I figure out that Postgrex.Interval was the default structure to decode PostgreSQL's intervals came through several researches about how to solve the error above

Learning is a constant in the software industry, however is not always possible to spend time as one would wish to acquire new knowledge, explore new libraries or frameworks, etc. Quite often developer seek for shortcuts that could solve the issue and allow them to move forward in a faster way

I've search exhaustively but I couldn't find nothing like a copy & paste to commit my code and end my day, so I start collecting pieces of information here and there stopping of being a lazy developer and reading more thedocumentation perhaps it may save some time from someone on future

I am wondering that I might not have found more info 'cause I didn't use the right keywords or by insisting to use DuckDuckGo as it sounds the results lately are poorer than before

Configuration

In a nutshell the configuration of Duration as interval decoding is done in two main files: config.exs e postgres_types.ex (the last one might have any name, postgres_types.ex is just a natural suggestion)

The file postgres_types.ex contains the configuration to overwrite the default one Postgrex.Interval, looking to file config.exs it associates the custom type to Ecto.Repo

Let's move forward and prepare the so expected snippet of code to copy, paste and close the day, thus the configuration that makes everything to work in a almost magical way (almost magical because in the end of day it's code design made by awesome guys behind the language, libraries and frameworks)is shown below:

# config/config.exs
# configures postgrex types

import Config

config :my_app,
  ecto_repos: [MyApp.Repo],
  generators: [timestamp_type: :utc_datetime]

# configures postgrex types
config :my_app, MyApp.Repo, types: MyApp.PostgresTypes

# Configures the endpoint
(config continues...)

# lib/my_app/postgres_types.ex
# set the interval decode

Postgrex.Types.define(
  ConstruaApi.PostgresTypes,
  [] ++ Ecto.Adapters.Postgres.extensions(),
  interval_decode_type: Duration
)
# --- that's all folks ---

It works

The search for a simple example, straight to the point doesn't work for me and in my rush to figure out the solution I have forgotten to ask to the available AI mechanisms on how to solve the issue in an easy and fast way (Doh!)

Now when someone seek for a handy code to solve this problem or a similar one here they have, but it's important to not forget (like me) the GPTs, Geminis and other AIs around to help

a pt_BR version is available here

Ctrl+Alt+Arrow (Right, Left) not working on IntelliJ

Rogério Ramos — Wed, 11 Dec 2024 13:36:11 +0000

This post aims to be just a quick tip for those facing the same issue as I was facing.
It seems just a stupid and not critical issue but it affects my productivity quite a lot and searching around I couldn't find a help directly to my problem but combining tips and and there I was able to figure out my issue

Lately I decide to refresh all my laptops to GNU Arch Linux installations before I used to have Kali, Manjaro and Arch. Now I have streamlined my installations and yeah the family at home had to overcome any barrier of using a GNU Linux box, even my little that is going to turn seven. For my surprise one of the my shortcuts were not working as usually does before

Going directly to the problem the configurations on my system was hiding some shortcut configs so it took me some time to realize that something was not correct
There is a very common shortcut conflict between Gnome and IntelliJ (when using the Xwin keymap). Gnome workspace switch to left|right (Settings -> Keyboard -> Keyboard Shortcuts) shares the same key combination* as IntelliJ (Xwin) navigate back and forward (File -> Navigate -> Back|Forward) which is the subject of this post

As one can see the configuration only shows one shortcut assigned to switching workspaces, when in fact 3 was set for moving to right and other 3 to left. The only way to discover it was to follow some tips like here and check the shortcuts under the table

user> $ gsettings get org.gnome.desktop.wm.keybindings \
switch-to-workspace-left
['<Super>Page_Up', '<Super><Alt>Left', '<Control><Alt>Left']

user> $ gsettings get org.gnome.desktop.wm.keybindings \
switch-to-workspace-right 
['<Super>Page_Down', '<Super><Alt>Right', '<Control><Alt>Right']

So as one can realize the configuration is shown in the screen only the first shortcut <Super>Page_Down and <Super>Page_Up but in the end other shortcuts were there hidden and provoking the malfunctioning behavior

Fixing is pretty easy when you know the reason, so just matter of remove/add the desired shortcuts

user> $ gsettings set org.gnome.desktop.wm.keybindings \
switch-to-workspace-right "['<Super>Page_Down', '<Super><Alt>Right']"

user> $ gsettings set org.gnome.desktop.wm.keybindings \
switch-to-workspace-left "['<Super>Page_Up', '<Super><Alt>Left']"

The final result was that everything works as expected and may be the next step suggest or even add contributions to Gnome project to have all shortcuts displayed in the configuration to prevent issues like this

* Ctrl+Alt+Left, Ctrl+Alt+Right

Migrating Spring Cloud Stream v2 to v3

Rogério Ramos — Fri, 21 Jan 2022 20:25:09 +0000

Cloud Stream V2 x V3

If you have landed here there is a high chance you are looking for ways of migrating the Spring Cloud Stream (SCS) to version 3.x where almost everything has changed

The version 3.x has its focus mainly in serverless applications where a service will execute a single function and the combination of all functions perform a workflow or process related to a given feature through Consumer, Supplier and/or Function bean functions, but not all applications are already using the new approach although is a trending now

Looking at version 2.x Sink, Source e StreamListeners build the set of configurations that allows an application to consume messages from multiple upstream sources to execute a flow as well as to publish messages with the resulting output of that flow

This post aims to demonstrate how the migration from version 2.x to 3.x can be done

The idea behind Cloud Function (CF) along with SCS is to be used in a FaaS application as unique function with single responsibility

A non-FaaS application can consumes information from several sources, that is an approach quite common on SCS v2.x, below it will be demonstrated how you can do such migration to SCS v3.x, if you have already checked what's new on SCS version 3 you will note that definitions of Sink, Source and Processor were replaced by Consumer, Supplier and Function bean functions

The function name mapping to a topic or queue might looks unclear at first sight, on SCS v2.x this used to be configure explicitly and it was easy to identify which topic or queue would be consumed by the application via Sink configuration and now you have to map the function name directly to the input or output configured on application.yaml

Particularly I didn´t like that much this configuration but I do not discard being only an initial impression or the natural resistance to novelties, depending on number of topics/queues it may produce a big and unreadable list of mapping functions (spring.cloud.stream.function.definition) on configuration file. I have not checked the limit - if any - about the size of this property value

Configuration

The application.yaml file has a mixture of Spring Cloud Stream and Spring Cloud Function configurations, the function and function definition belongs to Spring Cloud Function configuration

There isn't big differences in the configuration except the spring.cloud.stream.function.definition property that contains the functions to be mapped to bean functions Consumer, Supplier and Function names, the standard suffix for input and output configuration are <function-name>-in-0 and <function-name>-out-0

Consuming messages

On version 2 the Sink is used to configure which topic/queue will be listened for input messages as well as the annotation StreamListener that allows many configurations including to filter messages from upstream systems
On version 3 a Consumer function annotated with @Bean is used and the filter that was defined before via annotation now is done programmatically

Producing messages

The recommended method to produce messages is quite similar to the consumer function using Supplier instead Consumer, but an easier option in my opinion is to use the StreamBridge component that allows to send messages dynamically just passing the destination output

CloudEvents (optional)

Cloud Events is heavily used message specification and its usage is growing very fast, basically it puts a message payload in an "envelope" enriching the message with attributes like source, publish time, type, content type, etc

In the example below it's possible to check the event serialization and deserialization. Keep in mind that using CloudEvents is completely optional

The CloudEvents SDK offers many amenities that can be found here, e.g. the converter below allowing integration with Spring:

Conclusion and next steps

The goal here is to help with examples and configurations what has changed among SCS versions 2.x e 3.x and how such upgrade can be done

It's an abrupt change on how CSC is used and it can cause resistance to adopt keeping the already deprecated programming model still in use for while in the other hand can impose some challenging on running applications that will require a consistent set of testing mainly load and resiliency tests.

Tests were not covered on that time but they also changed quite a lot and it will be covered on further posts but if you can not wait to make it happen here you will find the steps to move forward

I hope this info could be helpful and don't hold yourself to share it everywhere

Esse post foi originalmente publicado aqui em português brasileiro

cya

-Rogério

The terrifying mongoDB sintaxe

Rogério Ramos — Fri, 07 Jun 2019 08:00:38 +0000

The main responsibility of experienced Sw Developers is help others and use the gained experience to overcome issues always teaching, learning and sharing.

I want to share a simple thing that make me think about complexity and trade-offs. I co-worker came to me asking for help on a MongoDB aggregate query:

How would I aggregate the data that I need since I am using a NoSQL database?

Can I use my previous SQL knowledge to make it happen?

Well, I think the things could be simpler but not. Look at the example bellow.

I have a series of events happening in my application where a particular event seems to be happening more often than expected. I would like check how often that event happens.

On mongo:

mongo-client-shell> db.orders.aggregate([ 
  { $match: {"event": "order.event.OrderPickedEvent"}} ,
  { $group : { _id: {"event": "$event", "orderNumber": '$orderNumber'}, count: { $sum: 1 } } }, 
  { $sort: {count: -1}}
], 
{allowDiskUse:true}
)

On Postgresql:

psql> select count(*) as total 
  from order 
  where event = "order.event.OrderPickedEvent" 
  group by orderNumber 
  order by total desc;

The question on my mind: Why Mongo make things so weird to not say complex?

Its not natural to the reader.
We are always talking about code clearness, readability and making things simpler. But seems that big player not thing in the same way.

I hope this mini post can help others and maybe sensitize the tool's suppliers to make life easier*.

I find a SQL translator to NoSQL which could help in that case https://studio3t.com/