DEV Community: Ronny Medina

The hidden costs of AI in development: a personal reflection

Ronny Medina — Wed, 27 May 2026 01:22:44 +0000

A personal reflection on the hidden costs of using AI in development — from the subscription you don't fully use, to the thousands of dollars burned on requirements that never make it to production.

A few months ago I started using Claude intensively on a personal project. I began with the 20-dollar-a-month plan and, as you'd expect, within a few weeks I was hitting limits. The caps came fast and kept cutting into my workflow. So I upgraded to the 100-dollar Max plan.

The initial spike: when everything seems to justify itself

When you start a project from scratch, there's a huge amount of "mechanical" work: configuration, scaffolding, base code, folder structure, first integrations, authentication boilerplate, models, migrations. That's the kind of work where AI really shines, because you can throw concrete tasks at it one after another without pause.

In that phase, the credits from the new plan were being used to the max. Productivity went up, progress felt real, and paying more seemed like the obvious choice.

But then the second phase arrived. And that's where I started to see the problem.

The bottleneck is me

As the project advanced, the blocker stopped being the AI's speed and started being me. I'm the one who defines the requirements. I'm the one thinking about what feature to add, how it should behave, what edge cases to consider. And that takes time. Time to read, to think, to test, to change my mind.

The result: I spend days when I'm not even using 50% of the credits I'm paying for. I'm paying for capacity I don't consume.

This reminded me of a fairly well-known concept in reactive programming: backpressure in streams. Backpressure happens when the producer sends data faster than the consumer can process it. In my case, the AI is the producer (it can generate code almost instantly) and I'm the consumer (I need time to define the next step).

The gym trap

There comes a point in every project where the number of requirements starts to drop. The architecture is in place, the core functionality exists, and what's left are details, refinements, and edge cases. At that point, keeping a high-tier subscription stops being worth it.

It's the same logic as a gym membership: you pay for a full month but only go two or three times. It's not that the gym is bad — your actual usage just doesn't justify the price.

We can imagine this same process inside companies. In an enterprise, by the time the Product Manager hands down a requirement, the dev understands it, discusses it, implements it, and reviews it, hours and sometimes days have gone by — and AI credits are just sitting there, waiting.

The new cost of cancelled projects

Something that happens in development: you start a feature, you make some progress, and halfway through the requirements change or the project gets cancelled outright. Before, the cost of that was mostly time. The dev got paid their salary anyway, but you could pivot without a direct financial hit.

Today it's different.

Picture this scenario: a developer is assigned a new requirement and, thanks to AI, instead of taking three months they ship it in two weeks. But in those two weeks they burned 3,000 or 4,000 dollars in credits. And it turns out that a month later the product changes direction and that code gets discarded. Before, you'd lose development time. Now you lose development time plus money in credits that you can't get back.

The worst part is that speed invites you to skip the analysis. If you produce fast, you spend less time asking yourself whether it's actually worth producing. And because everything feels instant, no one stops to ask whether the requirement was properly framed in the first place.

Speed hides errors that you pay for later

The last point, and maybe the most uncomfortable: AI generates code fast, but it also generates more errors. Not because AI is bad, but because the requirements we hand it are rarely 100% polished.

When you developed something manually, there was a natural process of rethinking the problem as you coded. You'd sit down to write a function and halfway through you'd realize something was missing, that an edge case wasn't covered, that there was a better way to model the problem. That process is valuable. It's where you find things that weren't in the requirements document.

With AI, that space to re-analyze the same problem multiple times gets compressed or disappears. You define a requirement, the AI ships functional code, the tests pass. And why do the tests pass? Because they cover exactly what you said they had to cover. The edge cases you didn't even think of don't show up in the tests either.

It happened to me on this project. When I sat down to review everything I'd generated 100% with AI, I found things that "worked" but had security issues, unhandled edge cases, and fragile logic in places that looked trivial. And tracing all of that when there are many features at once is really hard.

And here's the expensive part: you end up using the same AI, spending more credits, to review and fix the code that same AI generated. Ten thousand lines that then need two thousand more AI calls to audit, correct and validate.

My take

If I put it all together, in my personal experience I saw at least four clear ways to lose money with AI in development:

Human backpressure: you pay for capacity you can't consume because you are the bottleneck.
The gym trap: the high-tier subscription stops being worth it when requirements drop, but you keep paying for it.
More expensive cancellations: projects that get scrapped mid-way now take thousands of dollars in consumed credits with them, not just time.
Speed without analysis: you generate more code, with more errors, and end up spending more credits reviewing and fixing what the AI itself produced.

These days, many companies are adjusting their business models to be more efficient in their use of AI, especially with the rising costs we're seeing right now.

I'd love to hear other people's experiences.

El backpressure humano: por qué la IA puede estar costándote más de lo que produce

Ronny Medina — Wed, 27 May 2026 01:06:38 +0000

Una reflexión personal sobre los costos ocultos de usar IA en desarrollo, desde la suscripción que no aprovechas hasta los miles de dólares que se queman en requerimientos que nunca llegan a producción.

Hace unos meses empecé a usar Claude de forma intensiva en un proyecto personal. Arranqué con el plan de 20 dólares al mes y, como era de esperar, en pocas semanas me quedé corto. Los límites llegaban rápido y cortaban el flujo de trabajo. Así que subí al plan Max de 100 dólares.

El pico inicial: cuando todo parece justificarse

Cuando inicias un proyecto desde cero, hay una cantidad enorme de trabajo "mecánico": configuración, scaffolding, código base, estructura de carpetas, primeras integraciones, boilerplate de autenticación, modelos, migraciones. Es ese tipo de trabajo donde la IA realmente brilla, porque puedes ir tirándole tareas concretas una atrás de la otra sin pausa.

En esa fase, los créditos del nuevo plan se aprovechaban al máximo. La productividad subió, el avance se sentía real, y la decisión de pagar más parecía obvia.

Pero después llegó la segunda fase. Y ahí empecé a ver el problema.

El cuello de botella soy yo

A medida que el proyecto avanzó, el bloqueo dejó de estar en la velocidad de la IA y empezó a estar en mí. Yo soy quien define los requerimientos. Yo soy quien piensa qué función agregar, cómo debería comportarse, qué casos contemplar. Y eso lleva tiempo. Tiempo de leer, de pensar, de probar, de cambiar de idea.

Resultado: paso días en los que no estoy usando ni el 50% de los créditos que pago. Estoy pagando por capacidad que no consumo.

Esto me hizo acordar a un concepto bastante conocido en programación reactiva: el backpressure en streams. El backpressure aparece cuando el productor envía datos más rápido de lo que el consumidor puede procesarlos. En mi caso, la IA es el productor (puede generar código casi instantáneamente) y yo soy el consumidor (necesito tiempo para definir el próximo paso).

La trampa del gimnasio

Hay un punto en todo proyecto donde la cantidad de requerimientos empieza a bajar. La arquitectura ya está, las funcionalidades core ya existen, ahora son detalles, refinamientos, casos borde. En ese momento, mantener una suscripción de alto tier deja de ser rentable.

Es la misma lógica que la membresía del gimnasio: pagás por un mes entero pero vas dos o tres veces. No es que el gimnasio sea malo, es que tu uso real no justifica el precio.

Podemos imaginar este mismo proceso en empresas. En una empresa, entre que el Product Manager baja el requerimiento, el dev lo entiende, lo discute, lo implementa y lo revisa, pasan horas y a veces días donde los créditos de IA simplemente están ahí, esperando.

El nuevo costo del proyecto cancelado

Algo que en desarrollo pasa: Inicias una funcionalidad, avanzas un poco, y a mitad de camino cambia el requerimiento o directamente se cancela el proyecto. Antes, el costo de eso era principalmente tiempo. El dev cobraba su sueldo igual, pero podías cambiar el rumbo sin un golpe económico directo.

Hoy es distinto.

Imagina este escenario: a un desarrollador se le asigna un nuevo requerimiento y, gracias a la IA, en lugar de tardar tres meses lo tira en dos semanas. Pero en esas dos semanas quemó 3.000 o 4.000 dólares en créditos. Y resulta que al mes el producto cambia de dirección y ese código se descarta. Antes perdías tiempo de desarrollo. Hoy pierdes tiempo más dinero en créditos que ya no se recuperan.

Lo peor es que la velocidad invita a saltarse el análisis. Si producís rápido, te tomas menos tiempo para pensar si realmente vale la pena producir eso. Y como todo se siente instantáneo, nadie frena para preguntarse si el requerimiento estaba bien planteado desde el inicio.

La velocidad esconde errores que después se pagan

El último punto, y quizás el más incómodo: la IA genera código rápido, pero también genera más errores. No porque la IA sea mala, sino porque los requerimientos que le pasamos rara vez están al 100% pulidos.

Cuando desarrollabas algo manualmente, había un proceso natural de analizar nuevamente el problema mientras codeabas. Te sentabas a escribir una función y a mitad de camino te dabas cuenta de que faltaba algo, que un caso borde no estaba contemplado, que había una mejor forma de modelar el problema. Ese proceso es valioso. Es donde aparecen los hallazgos que no estaban en el documento de requerimientos.

Con la IA, ese espacio de analizar varias veces el mismo problema, se comprime o desaparece. Definís un requerimiento, lo tira y sale código funcional, los tests pasan. ¿Y por qué pasan los tests? Porque cubren exactamente lo que dijiste que tenían que cubrir. Los casos borde que ni vos pensaste, tampoco aparecen en los tests.

A mí me pasó en este proyecto. Cuando me senté a revisar todo lo que había generado al 100% con IA, encontré cosas que funcionaban "bien" pero que tenían problemas de seguridad, casos no contemplados, lógica frágil en partes que parecían triviales. Y trazar todo eso cuando son muchas funcionalidades a la vez es muy difícil.

Y acá viene la parte cara: terminas usando la misma IA, gastando más créditos, para revisar y arreglar el código que esa misma IA generó. Diez mil líneas que después necesitan dos mil llamadas más a la IA para auditar, corregir y validar.

Mi perspectiva

Si lo pongo todo junto, en mi experiencia personal vi al menos cuatro formas claras de perder dinero con IA en desarrollo:

Backpressure humano: pagas capacidad que no puedes consumir porque vos sos el cuello de botella.
La trampa del gimnasio: la suscripción de alto tier deja de ser rentable cuando los requerimientos bajan, pero seguís pagándola.
Cancelaciones más caras: los proyectos que se descartan a mitad de camino ahora se llevan miles de dólares de créditos consumidos, no solo tiempo.
Velocidad sin análisis: generas más código, con más errores, y terminas gastando más créditos revisando y arreglando lo que la IA misma produjo.

Hoy en día, muchas empresas están ajustando sus modelos de negocio para poder ser más óptimas en este proceso de uso de IA, más aún con la subida de costos que se está presentando.

Me encantaría leer la experiencia de otros.

Recomendaciones si vas a trabajar con BigQuery

Ronny Medina — Mon, 09 Sep 2024 03:16:16 +0000

¡Hola a todos! Espero que se encuentren bien. En este post, quiero compartir mi experiencia trabajando con BigQuery y los aprendizajes que obtuve en el proceso.

Hace un tiempo, me enfrenté a la tarea de sincronizar los datos de una base de datos a BigQuery. El reto no solo era trasladar la información desde una base de datos relacional, sino hacerlo de manera que los datos estuvieran más procesados y optimizados para su uso en BigQuery

Cómo aprovechar BigQuery sin encasillarlo en el modelo relacional o no relacional

En este punto, lo que quiero decir es que no trates de llevar al 100% la misma estructura que tienes en tu base de datos relacional o no relacional. No quiero decir que esté mal; va a depender de cada caso. Desde mi perspectiva, intentar trasladar esta misma relación no genera ningún beneficio en BigQuery que sea significativo, o al menos no desde mi conocimiento, ya que algunas relaciones que hacemos son para poder optimizar consultas y evitar duplicar datos en tablas.

Como en la imagen anterior, tal vez no sea necesario intentar trasladar la tabla transaction_types, ya que esta solo sirve para evitar duplicar datos en una misma tabla.

BigQuery te permite trabajar con datos estructurados y no estructurados, y se define como un servicio de almacenamiento.

Puntos relevantes:

No soporta transacciones
No existen índices para hacer relaciones o tener campos únicos
Permite el uso de SQL para manejar operaciones
También puedes modelar los datos como si trabajaras en Mongo

Si puedes saber de antemano cómo se va a operar con los datos, esto te será de gran ayuda al momento de trasladar la información. En ocasiones, no es posible conocer cómo se manejarán los datos al 100%, ya que pueden ser equipos externos quienes los manipulen con otras operaciones.

Comprender los costos operativos de BigQuery

Es importante entender los costos operativos, ya que BigQuery aplica diferentes tipos de cobros según las operaciones que se realicen. Dependiendo de lo que se quiera llevar a cabo, es fundamental tener estos costos en cuenta, ya que pueden ayudar a optimizar las operaciones y evitar gastos innecesarios. Suponiendo que solo queremos almacenar, consultar y transferir los datos, podemos destacar lo siguiente:

Costo por almacenamiento

Costo por consultas, que incluye la definición de esquemas y la manipulación de datos

Costo por transferencia de datos

Te recomiendo consultar la página de precios de BigQuery para obtener información detallada y realizar un mejor análisis e investigación de los costos.

Puntos relevantes:

Tener presente los costos y saber como vas operar

Riesgos de la simplificación excesiva en la estructura de tablas

Como mencioné anteriormente sobre no trasladar directamente la estructura de tu modelo de base de datos a BigQuery, en el proceso de intentar simplificar o desnormalizar los datos, podríamos caer en el error de complicar el manejo de la información. Por ejemplo, en mi caso, tenía una situación similar, donde había una relación de uno a muchos.

more....

Configuración de Dockerfile para proyecto Nest.js: tutorial práctico

Ronny Medina — Sat, 22 Jun 2024 02:54:04 +0000

Post original .

Hola a todos, en este post quiero hacer una actualización de mi post anterior en Nest.js. Quiero dejar una configuración básica para un proyecto nuevo con este framework usando Docker.

Requerimientos

Tener Nodejs
Tener Docker
Conocimientos basicos en Docker

Instalación

El primer comando que necesitamos usar es el siguiente, que se encuentra en la página oficial, para poder instalar la CLI en nuestro sistema operativo y poder generar el proyecto.



$ npm i -g @nestjs/cli
$ nest new project-name

Creando nuestro Dockerfile

Lo primero que necesitamos es crear un archivo que se encargue de levantar nuestra app en modo productivo o desarrollo. Este archivo se llamará run.sh



#!/bin/bash
echo "NODE_ENV: $NODE_ENV";

if [ "$NODE_ENV" == "prod" ]
    then
        echo "start prod mode...";
        npm run start:prod
else
    echo "start dev mode...";
    npm run start:dev
fi

Este archivo lee la variable de entorno NODE_ENV y, en caso de ser el valor «prod», ejecutará el comando para correr en modo productivo; caso contrario, ejecutará nuestra app en modo desarrollo.

Luego de crear este archivo, vamos a crear nuestro archivo Dockerfile.



FROM node:20.14.0-bullseye

ENV APP_PORT 3000
ENV NODE_ENV prod
ENV WORKDIR_APP /var/prod

WORKDIR ${WORKDIR_APP}
COPY package.json .
RUN npm install
COPY . .

RUN npm run build

EXPOSE ${APP_PORT}

CMD ["bash", "run.sh"]

Configurando nuestra app

En nuestro proyecto vamos a crear una carpeta llamada config y dentro un archivo llamado envs.ts. Este archivo tendrá nuestras variables de entorno de nuestra aplicación.



export const APP_PORT = parseInt(process.env.APP_PORT, 10);
export const APP_IS_PROD = process.env.NODE_ENV === 'prod'

Luego vamos a configurar nuestro main.ts. Vamos a definir una variable llamada logLevel. Esta contendrá los niveles de log por defecto en modo productivo y agregaremos los otros en modo desarrollo. Esto es completamente configurable a sus necesidades.

Adicionalmente activaremos el versionamiento de nuestra API.




import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { APP_IS_PROD, APP_PORT } from './config/envs';
import { LogLevel, Logger, VersioningType } from '@nestjs/common';

async function bootstrap() {
  const logger = new Logger('bootstrap');
  const logLevel: LogLevel[] = ['error', 'warn', 'fatal', 'log'];

  if (!APP_IS_PROD) {
    logLevel.push('debug', 'verbose');  }


  const app = await NestFactory.create(AppModule, { logger: logLevel });

  app.enableVersioning({type: VersioningType.URI});

  await app.listen(APP_PORT);
}
bootstrap();

Construir nuestra imagen

Ahora podemos correr los siguientes comandos para construir nuestra imagen.




$ docker build -t nestjs-setup .
$ docker run -it --rm -e NODE_ENV=develop -p "3000:3000" --name nest-setup nest-setup

Ahora debemos poder acceder al puerto 3000 y ver el mensaje de Hello World!.

Agregando validaciones & Swagger

Adicionalmente voy a agregar los paquetes para las validaciones y para documentar nuestra API.

Para eso debemos entrar en nuestro contenedor que esta corriendo el siguiente comando:




$ docker exec -it nest-setup bash

Agregar las siguientes dependencias:

Swagger
Class validator




$ yarn add class-validator class-transformer
$ yarn add @nestjs/swagger

Para este punto, debemos actualizar nuestro package.json, ya que los cambios se realizaron dentro del contenedor y no estamos usando ningún volumen en este momento. Podemos copiar el archivo package.json desde nuestro contenedor usando docker cp. No es necesario que realice este paso, ya que al final del post tendrá todo el código con todas las dependencias.

Ahora en nuestro main vamos a agregar una configuración adicional para poder ver los errores de las validaciones cuando estemos en modo desarrollo. Este paso para ver los errores es opcional pero puede ser útil si desea aplicar otro caso de uso.




import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { SwaggerModule, DocumentBuilder } from '@nestjs/swagger';
import { APP_IS_PROD, APP_PORT } from './config/envs';
import { BadRequestException, LogLevel, Logger, ValidationError, ValidationPipe, VersioningType } from '@nestjs/common';

async function bootstrap() {
  const logger = new Logger('bootstrap');
  const logLevel: LogLevel[] = ['error', 'warn', 'fatal', 'log'];

  if (!APP_IS_PROD) {
    logLevel.push('debug', 'verbose');
  }

  const app = await NestFactory.create(AppModule, { logger: logLevel });

  app.enableVersioning({type: VersioningType.URI});

  const config = new DocumentBuilder()
    .setTitle('Cats example')
    .setDescription('The cats API description')
    .setVersion('1.0')
    .addTag('cats')
    .build();

  const document = SwaggerModule.createDocument(app, config);
  SwaggerModule.setup('docs', app, document);

   // print validation errors
   app.useGlobalPipes(
    new ValidationPipe({
      enableDebugMessages: !APP_IS_PROD,
      transform: true,
      exceptionFactory: (errors: ValidationError[]) => {
        const allErrorMsg = errors.flatMap((error) =>
          Object.values(error.constraints),
        );

        if (!APP_IS_PROD) {
          logger.debug('Validations errors', allErrorMsg);
          logger.debug('Error schema', errors);
        }

        return new BadRequestException(allErrorMsg);
      },
    }),
  );


  await app.listen(APP_PORT);
}
bootstrap();

Modificando nuestro app.controller.ts.




import { Body, Controller, Get, Post } from '@nestjs/common';
import { IsString } from 'class-validator';
import { ApiProperty } from '@nestjs/swagger';
import { AppService } from './app.service';

export class AppDto {
  @ApiProperty({ example: 'demo' })
  @IsString()
  name: string;
}

@Controller({
  version: '1',
  path: 'demo',
})
export class AppController {
  constructor(private readonly appService: AppService) {}

  @Post()
  getHello(@Body() data: AppDto): string {
    return this.appService.getHello();
  }
}

Ahora podemos acceder a la siguiente URL: http://localhost:3000/docs. Al enviar la solicitud, podemos ver que los datos se envían correctamente.

Si enviamos un número al momento de realizar la solicitud, podemos ver los mensajes de error también en la terminal.

Docker compose

Por ultimo podemos agregar nuestro docker-compose.yml



version: '3'

services:

  nest-setup:

    container_name: nest-setup

    build: .

    ebvironment:

      - NODE_ENV=develop

    ports:

      - "3000:3000"

    volumes:

      - ./src:/var/prod/src

      - ./test:/var/prod/test

      - ./package.json:/var/prod/package.json



$ docker compose up

Recursos

Golang Logging Configuration with Zap: Practical Implementation Tips

Ronny Medina — Fri, 17 May 2024 14:03:14 +0000

Hi everybody, continuing from the previous post, which was a basic setup with Golang. In this part, we're setting up our logging in the app.

THE ORIGINAL POST HAS ADDITIONAL INFORMATION.

Installing the package zap

https://github.com/uber-go/zap

In our project, we need to install the zap package. To do this, put the following command in our terminal.

go get -u go.uber.org/zap

If you're using Docker like me, you need to enter the container. So, if you're running the app in the background, execute this command.

docker exec -it demogolang bash

To run the app in the background, use the command: docker-compose up -d.
The name 'demogolang' is declared inside the docker-compose.yml file.

Setting up the logging

Dockerfile

In our Dockerfile add 2 environment variables and created a folder to save the logs.

APP_LOG_LEVEL: The app log level could be debug or error. We use the debug level when we're in development and error when the app is in production mode. So, if you're in error mode when you print logs Info or Debug, these are not shown in the terminal. If you want to use more levels, you can edit the function to get the current level.
APP_LOG_FOLDER: This is the path where you place the log file. You can set whatever path you want.


ENV APP_LOG_LEVEL debug
ENV APP_LOG_FOLDER /tmp/logs/

RUN mkdir -p ${APP_LOG_FOLDER}

Reading environment variables

Inside our app folder, I created a folder called config. Inside config, I put the file called envs.go.


package config

import "os"

var APP_ENV = os.Getenv("APP_ENV")
var IS_DEVELOP_MODE = APP_ENV == "develop"
var APP_LOG_LEVEL = os.Getenv("APP_LOG_LEVEL")
var APP_LOG_FOLDER = os.Getenv("APP_LOG_FOLDER")

Logger file


package utils

import (
    "example/demogo/config"

    "go.uber.org/zap"
    "go.uber.org/zap/zapcore"
)

var Logger *zap.SugaredLogger

func getLevelLogger(level string) zapcore.Level {
    if level == "debug" {
        return zap.DebugLevel
    }

    return zap.ErrorLevel
}

func init() {
    var err error
    level := zap.NewAtomicLevelAt(getLevelLogger(config.APP_LOG_LEVEL))
    encoder := zap.NewProductionEncoderConfig()

    zapConfig := zap.NewProductionConfig()
    zapConfig.EncoderConfig = encoder
    zapConfig.Level = level
    zapConfig.Development = config.IS_DEVELOP_MODE
    zapConfig.Encoding = "json"
    zapConfig.InitialFields = map[string]interface{}{"idtx": "999"}
    zapConfig.OutputPaths = []string{"stdout", config.APP_LOG_FOLDER + "app_log.log"}
    zapConfig.ErrorOutputPaths = []string{"stderr"}
    logger, err := zapConfig.Build()

    if err != nil {
        panic(err)
    }

    Logger = logger.Sugar()
}

I created a global variable called Logger; this will be set when the package is initialized. The function getLevelLogger is used to get the current level from the environment variable.

The init function is automatically called before the main function.

encoder: Is the log format, how you want to apply the format.
InitialFields: The values that you want to always be in the log.
OutputPaths: The output of the file.
ErrorOutputPaths: To print error log in the terminal.

Finally our main.go.


package main

import (
    "example/demogo/utils"
    "time"
)

func main() {
    defer utils.Logger.Sync()

    for {
        time.Sleep(5 * time.Second)
        utils.Logger.Infoln("hello world")
    }
}

It's important to call defer utils.Logger.Sync() in the entry point of our app. Normally ensures that all buffered log messages are sent to their final destination if the application crashes.

docker-compose up -d --build

docker logs -f demogolang

Basic setup for projects with Golang

Ronny Medina — Tue, 14 May 2024 02:08:14 +0000

Post original in my blog.

PART 2

Hi everybody! In this post, I'm going to explain how to create a Golang project with Docker. Video en español en mi canal.

This repository contains the code.

Requirements

Docker installed
Basic knowledge with docker
Docker compose installed

Structure project

The structure for this project is very simple.

app: Contains all the files of our application.
main.go: Application entry point.
Dockerfile: Dockerfile to build our image.
docker-compose.yml: File to help build our image more easily.
go.mod: File to track your project dependencies.

your-folder-project
    app
       main.go
       run.sh
       go.mod
    Dockerfile
    docker-compose.yml

Dockerfile

FROM golang:1.22.3-bullseye

ENV APP_ENV prod
ENV WORDIR_APP /usr/src
ENV BINARY_NAME main
ENV BINARY_PATH ${WORDIR_APP}/${BINARY_NAME}

WORKDIR ${WORDIR_APP}
COPY app/go.mod app/run.sh ./

RUN go mod download && \
    go mod verify && \
    go install github.com/cosmtrek/air@v1.51.0

WORKDIR ${WORDIR_APP}/app

COPY ./app .

RUN go build -v -o ${BINARY_PATH}

CMD [ "bash", "run.sh" ]

In this file we define image version FROM golang:1.22.3-bullseye. Then there are 4 environments.

APP_ENV: This environment is used to manage the mode in which our application is running. We use values like: (prod, develop, test).
WORDIR_APP: It is the location when we are working.
BINARY_NAME: This is the name of our application when it is finally built. You can use any name that you want.
BINARY_PATH: This is the full path of our binary when the application is built.

On line 11, we're installing all dependencies in our project, and additionally, we add this library to enable live-reload functionality in our application.

I think that was the most important thing in Dockerfile.

go.mod

This file is to track dependencies in your project. You can check the basic tutorial.

module example/demogo

go 1.22.3

module: The name of your package.

go: Golang version. In this case this version must be the same one that we use in our Dockerfile

run.sh

#!/bin/bash

echo "=== Start app in (${APP_ENV}) ==="

env=${APP_ENV:-}

if [ "$env" = "prod" ]; then
    # exucute binary
    ${BINARY_PATH} 
else
    air --build.cmd "go build -v -o ${BINARY_PATH}" --build.bin "${BINARY_PATH}"
fi

In this file, we have a conditional to execute a command depending on whether APP_ENV is prod or develop. If APP_ENV is prod, it executes the binary that we previously compiled in our Dockerfile. Otherwise, we execute the air command to use live reload in our app.

docker-compose.yml

version: '3'
services:
  demogolang:
    build: .
    environment:
      - APP_ENV=develop
    volumes:
      - ./app:/usr/src/app

Start app

To start our application we only have to execute the following command: docker compose up.

https://dev.to/ronnymedina/query-large-amounts-of-data-from-bigquery-faster-with-golang-4j61

Query large amounts of data from BigQuery faster with Golang

Ronny Medina — Sun, 29 Oct 2023 01:31:23 +0000

Original post from Edulixir.

Hi every one, In this posts I going to show you, how enable storage api. With this option, you can get big data results faster. I was working with BigQuery and in some cases I need to get big data results.

I'm going to assume that you already have the Google project with BigQuery active. For this example I'm going to use public dataset. It doesn't matter what data set you are using.

For example, this table has 540584 records.

Code without storage api


package main

import (
    "context"
    "fmt"
    "time"

    "cloud.google.com/go/bigquery"
    "google.golang.org/api/iterator"
    "google.golang.org/api/option"
)

var client *bigquery.Client
var ProjectId string
var CREDENTIALS_DIR = "credential.json"

func timeTracker(start time.Time, name string) {
    elapsed := time.Since(start)
    msg := fmt.Sprintf("==== %s -> time execute: %s ===", name, elapsed)

    fmt.Println(msg)
}

func connectClientToBq(ctx context.Context, projectId string) error {
    var err error
    path := CREDENTIALS_DIR

    credentialOpts := option.WithCredentialsFile(path)
    client, err = bigquery.NewClient(ctx, projectId, credentialOpts)
    if err != nil {
        return err
    }

    return nil
}

func closeClientBq() {
    ProjectId = ""

    if client != nil {
        client.Close()
    }
}

func executeQueryWithJob(ctx context.Context, sql string, params []bigquery.QueryParameter) (*bigquery.RowIterator, error) {
    defer timeTracker(time.Now(), "ExecuteQueryWithJob")

    q := client.Query(sql)
    q.Parameters = params
    job, err := q.Run(ctx)

    if err != nil {
        return nil, err
    }

    status, err := job.Wait(ctx)
    if err != nil {
        return nil, err
    }

    if err := status.Err(); err != nil {
        return nil, err
    }

    return job.Read(ctx)
}

func main() {
    defer closeClientBq()
    defer timeTracker(time.Now(), "Finish")
    var params []bigquery.QueryParameter
    ctx := context.Background()
    projectId := "demos-403423"
    err := connectClientToBq(ctx, projectId)

    if err != nil {
        fmt.Println(err.Error())
    }

    sql := "SELECT difficultyTarget, version, work_terahash, block_id FROM `bigquery-public-data.bitcoin_blockchain.blocks`"
    data, err := executeQueryWithJob(ctx, sql, params)

    if err != nil {
        fmt.Println(err.Error())
    }

    for {
        var values []bigquery.Value
        err := data.Next(&values)

        if err == iterator.Done {
            break
        }
        if err != nil {
            fmt.Println(err.Error())
        }
    }
}

For this query the execution time was:

==== ExecuteQueryWithJob -> time execute: 1.939979125s ===

The execution time to get all the data was:

==== Finish -> time execute: 1m30.853989167s ===

Storage API

Now if you enable the storage API, in the function connectClientToBq

func connectClientToBq(ctx context.Context, projectId string) error {
    var err error
    path := CREDENTIALS_DIR

    credentialOpts := option.WithCredentialsFile(path)
    client, err = bigquery.NewClient(ctx, projectId, credentialOpts)
    if err != nil {
        return err
    }

    err = client.EnableStorageReadClient(ctx, credentialOpts)
    if err != nil {
        return err
    }

    return nil
}

The times now were:

==== ExecuteQueryWithJob -> time execute: 3.695876083s ===

The execution time to get all the data was:

==== Finish -> time execute: 23.907911666s ===

I hope this post is useful to you. If you can share

Resources

Geolocation with Postgresql+Nestjs+Typeorm

Ronny Medina — Fri, 03 Dec 2021 12:41:13 +0000

Hi everyone. It's been a long time since I last posted something.

Some time ago I needed create an application that using geolocation to find ads nearby.

In my search, I take the easy solution which was to use Firebase.

Integration was easy because it was a new project. I did some testing and this worked great. I don't have much experience with Firebase.

Some problems I had were the following:

Try modeling tables in Firebase collections.
Thinking about the cost.
Difficulty trying to query multiple related collections

If I had more experience with Firebase this would be easy for me, but it is not the case.

So I found another solution that I think is easy to integrate.

Geo postgresql. You can visit the original publication and see in more detail all the explanations.

I did an example with Nestjs.

You can download this repository and run docker-compose up -d, this should get the server up. And you can access url /docs.

I hope this publication is helpful to you. I made a video tutorial in Spanish.

Publish/Subscribe pattern example (Redis, Kafka)

Ronny Medina — Sun, 18 Jul 2021 13:48:59 +0000

Hi, in tutorial I'm goin to show you how to implement this pattern. I won't explain details about Kafka or Redis. Then main idea here is to show you how to create simple implementation with these technologies.

To this tutorial we going to use Docker and docker-compose is necessary if you want to follow this tutorial.

Explain

The publish/subscribe pattern is very easy to understand. The main idea is to has central point where all the applications will put the information and then it will be consumed by other applications. You can see the imagen below.

Depends on the technologies we're using there're some terms that will change. For example in Redis to classify our information they call it a channel. In Kafka they call it topic.

Topics or channels is the way to classify information. For example, we will save information about the login process. We can create a (topic|chennel) called login_info and another to save then transaction information of a payment called transaction_payment.

Before start we need to install some dependencies.

npm install express ioredis kafkajs

docker-compose (Redis)

We need to create a docker-compose.yml and put this content.


version: "3.9"
services:
  redis-service:
    image: "redis:alpine"
    container_name: redis-service
    ports:
      - "6379:6379"

Publisher (Redis)

publisher.js. You can create an endpoint to send the information at moment. For me this approach works.

const express = require('express')
const Redis = require('ioredis')

const app = express()
const port = 3000
const redis = new Redis()

app.listen(port, async () => {
  const message = { message: 'message from redis' }
  redis.publish('test', JSON.stringify(message))
  console.log(`Example app (publisher) at http://localhost:${port}`)
})

Subscriber (Redis)

subscriber.js


const express = require('express')
const Redis = require('ioredis')

const app = express()
const port = 3500

const redis = new Redis()

app.listen(port, async () => {
  await redis.subscribe('test')

  redis.on('message', (channel, message) => {
    console.log(`Received ${message} from ${channel} (Redis)`)
  })

  console.log(`Example app (subscribe) at http://localhost:${port}`)
})

Now need to execute the following commands.


$ docker-compose up -d

$ node subscriber.js

$ node publisher.js // open in another terminal

You should now see a message in your terminal (subscriber). That was very easy =). Now we can do the same process to implement Kafka.

docker-compose (Kafka)

To implement Kafka we need to download a docker-compose.yml file from confluent page https://docs.confluent.io/platform/current/quickstart/ce-docker-quickstart.html.

We can use the entire docker-compose file with all services. But it is not necessary for this tutorial. The fundamental services to start working with Kafka are zookeeper and broker.


version: "3.9"
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:6.2.0
    hostname: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  broker:
    image: confluentinc/cp-server:6.2.0
    hostname: broker
    container_name: broker
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
      - "9101:9101"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_CONFLUENT_LICENSE_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_CONFLUENT_BALANCER_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_JMX_PORT: 9101
      KAFKA_JMX_HOSTNAME: localhost
      KAFKA_CONFLUENT_SCHEMA_REGISTRY_URL: http://schema-registry:8081
      CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: broker:29092
      CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1
      CONFLUENT_METRICS_ENABLE: 'true'
      CONFLUENT_SUPPORT_CUSTOMER_ID: 'anonymous'

Publisher (Kafka)


const express = require('express')
const { Kafka } = require('kafkajs')

const app = express()
const port = 3000


const kafka = new Kafka({
  clientId: 'my-app',
  brokers: ['localhost:9092']
})

app.listen(port, () => {
  const producer = kafka.producer()
  await producer.connect()
  await producer.send({
    topic: 'test',
    messages: [{ value: 'Hello KafkaJS user!' }],
  })

  await producer.disconnect()

  console.log(`Example app (publisher) at http://localhost:${port}`)
})

Subscriber (Kafka)

Before I begin, I need to clarify that this code could be anywhere you want.


  const admin = kafka.admin()

  try {
    await admin.connect()

    await admin.createTopics({
      waitForLeaders: true,
      topics: [{topic: 'test' }],
    })
  } catch(err) {
    console.error(err)
  } finally {
    await admin.disconnect()
  }

Kafka needs setup before starting. For example we need to create the topics before. Kafka has more settings unlike Redis. I put this content inside subscriber.js if you run this code multiple times this throws an error because the topic already exists.


const express = require('express')
const { Kafka } = require('kafkajs')

const app = express()
const port = 3500

const kafka = new Kafka({
  clientId: 'my-app',
  brokers: ['localhost:9092']
})

app.listen(port, async () => {
  const admin = kafka.admin()

  try {
    await admin.connect()

    await admin.createTopics({
      waitForLeaders: true,
      topics: [{topic: 'test' }],
    })
  } catch(err) {
    console.error(err)
  } finally {
    await admin.disconnect()
  }

  // kafka
  const consumer = kafka.consumer({ groupId: 'test-group' })
  await consumer.connect()
  await consumer.subscribe({ topic: 'test', fromBeginning: true })
  await consumer.run({
    eachMessage: async ({ topic, partition, message }) => {
      console.log(`Received ${message.value.toString()} from channel ${topic} (Kafka)`)
    },
  })

  console.log(`Example app (subscribe) at http://localhost:${port}`)
})

That's it to be able to work with Kafka. Now we can run the same commands we saw before.


$ docker-compose up -d

$ node subscriber.js

$ node publisher.js // open in another terminal

I hope it was useful for you and that you can start working with these technologies. If you have any corrections, let me know.

FastApi Example

Ronny Medina — Mon, 14 Jun 2021 02:35:28 +0000

Video en español.

Hello everyone, in this post I'm going to show you a small example with FastApi.

Well, to use FastApi, we need to install some dependencies such as:

pip install fastapi
pip install uvicorn[standard]

Or we can create a requirements file. requirements.txt.

fastapi==0.65.2
uvicorn==0.14.0

And then run the following command pip install -r requirements.txt.

The next step is to create a main file main.py and put the following content inside.

from fastapi import FastAPI

app = FastAPI()


@app.get('/')
def root():
    return 'ok'


@app.get('/item/{id}')
def read_item(id: int):
    return id

Now we can run the following command uvicorn main:app --reload.

We can check the urls in our browser http://localhost:8000/ and http://localhost:8000/item/1.

By default, FastApi has swagger included. We can check this url in your browser http://localhost:8000/docs. Another system is also integrated to document our api. You can visit http://localhost:8000/redoc.

The documentation of our api is automatic. This happens when we write the typing of our code. If you change this line def read_item(id: int) to def read_item(id: str) this updates our documentation.

To keep things in order we can create folder app and inside the following structure.

And inside the schemas we are going to create two files. A item.py this file is to save the validations of this resource.

from typing import Optional
from pydantic import BaseModel, Field

class Item(BaseModel):
    name: str =  Field(None, title='Example title...', min_length=3, max_length=10)
    price: float = Field(None, title='Price 300')
    is_offer: Optional[bool] = Field(False, title='Is offer')

You can visit the official page for more information pydantic-docs. Then we are going to create a __init__.py to export this validation.


from .item import Item as ItemSchema

Now we're going to create router file (router.py).


from typing import List
from fastapi import APIRouter

from .schemas import ItemSchema


router = APIRouter()

@router.get('/items', tags=['items'], response_model=List[ItemSchema])
def list_item() -> List[ItemSchema]:
    return []


@router.get('/items/{id}', tags=['items'])
def read_item(id: int) -> int:
    return id


@router.post('/items', tags=['items'], response_model=ItemSchema)
def create_item(data: ItemSchema):
    return data

Now we need to export this router to use it. You can create another __init__.py at the same level as the item folder.


from .router import router as item_router

And finally we need to update our main file.


from fastapi import FastAPI

from .items import item_router

app = FastAPI()

app.include_router(item_router)


@app.get('/')
def root():
    return 'ok'

To run our api we can execute this command uvicorn app.main:app --reload

Now we split our application and update the documentation.

How does validation work in FastApi?

This runs as a middleware if the data is invalid the return statement is never executed. You can try to pass invalid data to this API.


@router.post('/items', tags=['items'], response_model=ItemSchema)
def create_item(data: ItemSchema):
    return data

Well this was a small example with FastApi. I Hope this was helpful to you.

How to split your React app?

Ronny Medina — Mon, 05 Apr 2021 14:41:18 +0000

Hi everybody in this post I'm going to show you, how to split your React app. This post is very easy and quick to follow.

Why do you need to split your application?

To answer this question in two phrases, can be:

Gain performance in your application
Don't load unnecessary content

You could see this example (Before):

In that example, we loaded these icons asynchronously. Sometimes we have very heavy assets.

import React from 'react';

import DeleteIcon from '@material-ui/icons/Delete';
import DeleteOutlinedIcon from '@material-ui/icons/DeleteOutlined';
import DeleteRoundedIcon from '@material-ui/icons/DeleteRounded';
import DeleteTwoToneIcon from '@material-ui/icons/DeleteTwoTone';
import DeleteSharpIcon from '@material-ui/icons/DeleteSharp';
import DeleteForeverIcon from '@material-ui/icons/DeleteForever';
import DeleteForeverOutlinedIcon from '@material-ui/icons/DeleteForeverOutlined';
import DeleteForeverRoundedIcon from '@material-ui/icons/DeleteForeverRounded';
import DeleteForeverTwoToneIcon from '@material-ui/icons/DeleteForeverTwoTone';
import DeleteForeverSharpIcon from '@material-ui/icons/DeleteForeverSharp';
import ThreeDRotationIcon from '@material-ui/icons/ThreeDRotation';
import FourKIcon from '@material-ui/icons/FourK';
import ThreeSixtyIcon from '@material-ui/icons/ThreeSixty';

export const Home = () => {
  return (
    <div>
      <h1>Home</h1>
        <DeleteIcon />
        <DeleteOutlinedIcon />
        <DeleteRoundedIcon />
        <DeleteTwoToneIcon />
        <DeleteSharpIcon />
        <DeleteForeverIcon />
        <DeleteForeverOutlinedIcon />
        <DeleteForeverRoundedIcon />
        <DeleteForeverTwoToneIcon />
        <DeleteForeverSharpIcon />
        <ThreeDRotationIcon />
        <FourKIcon />
        <ThreeSixtyIcon />
    </div>
  )
}

As we can see, our page loaded in 19.47 s. In the example above, our page is blocked until all content is loaded.

Using `React.lazy`

import React, { Suspense } from 'react';

const DeleteIcon = React.lazy(() => import('@material-ui/icons/Delete'));
const DeleteOutlinedIcon = React.lazy(() => import('@material-ui/icons/DeleteOutlined'));
const DeleteRoundedIcon = React.lazy(() => import('@material-ui/icons/DeleteRounded'));
const DeleteTwoToneIcon = React.lazy(() => import('@material-ui/icons/DeleteTwoTone'));
const DeleteSharpIcon = React.lazy(() => import('@material-ui/icons/DeleteSharp'));
const DeleteForeverIcon = React.lazy(() => import('@material-ui/icons/DeleteForever'));
const DeleteForeverOutlinedIcon = React.lazy(() => import('@material-ui/icons/DeleteForeverOutlined'));
const DeleteForeverRoundedIcon = React.lazy(() => import('@material-ui/icons/DeleteForeverRounded'));
const DeleteForeverTwoToneIcon = React.lazy(() => import('@material-ui/icons/DeleteForeverTwoTone'));
const DeleteForeverSharpIcon = React.lazy(() => import('@material-ui/icons/DeleteForeverSharp'));
const ThreeDRotationIcon = React.lazy(() => import('@material-ui/icons/ThreeDRotation'));
const FourKIcon = React.lazy(() => import('@material-ui/icons/FourK'));
const ThreeSixtyIcon = React.lazy(() => import('@material-ui/icons/ThreeSixty'));


export const Home = () => {
  return (
    <div>
      <h1>Home</h1>
      <Suspense fallback={<div>Loading...</div>}>
        <DeleteIcon />
        <DeleteOutlinedIcon />
        <DeleteRoundedIcon />
        <DeleteTwoToneIcon />
        <DeleteSharpIcon />
        <DeleteForeverIcon />
        <DeleteForeverOutlinedIcon />
        <DeleteForeverRoundedIcon />
        <DeleteForeverTwoToneIcon />
        <DeleteForeverSharpIcon />
        <ThreeDRotationIcon />
        <FourKIcon />
        <ThreeSixtyIcon />
        </Suspense>
    </div>
  )
}

Now we can see the Home text before all the content is loaded (22.66s).

Our page takes a little longer to load but that's ok.

We can imagine that we have a list of users and two buttons. One button opens modal to add a new user and the other button is another modal to send feedback. So we can use React.lazy to load these modals. We can say that the main functionality is our user list. We don't need to lock the whole page for two modals that maybe the user does not use.

Split our routes

We can split routes, look at this example.

import React, { Suspense, lazy } from 'react';
import { BrowserRouter as Router, Route, Switch } from 'react-router-dom';
import Skeleton from '@material-ui/lab/Skeleton';

const Home = lazy(() =>
 import(/* webpackChunkName: 'Home' */'./components/home').then(m => ({ default: m.Home }))
);

function App() {
  return (
    <div className="App">
      <Router>
        <Suspense fallback={<Skeleton variant="rect" width={210} height={118} />}>
          <Switch>
            <Route exact path="/" component={Home}/>
          </Switch>
        </Suspense>
      </Router>
    </div>
  );
}

export default App;

I hope this post is helpful to you. If you have corrections please write me a comment. Thank you very much =)

Kafka introducción & implementación en Nodejs (node+express)

Ronny Medina — Mon, 01 Mar 2021 02:33:09 +0000

En este ejemplo veremos como implementar kafka y como podemos escribir y leer mensajes con nodejs. Puedes ver el video para tener información más detallada.

DEV Community: Ronny Medina

The hidden costs of AI in development: a personal reflection

The initial spike: when everything seems to justify itself

The bottleneck is me

The gym trap

The new cost of cancelled projects

Speed hides errors that you pay for later

My take

El backpressure humano: por qué la IA puede estar costándote más de lo que produce

El pico inicial: cuando todo parece justificarse

El cuello de botella soy yo

La trampa del gimnasio

El nuevo costo del proyecto cancelado

La velocidad esconde errores que después se pagan

Mi perspectiva

Recomendaciones si vas a trabajar con BigQuery

Cómo aprovechar BigQuery sin encasillarlo en el modelo relacional o no relacional

Comprender los costos operativos de BigQuery

Riesgos de la simplificación excesiva en la estructura de tablas

Configuración de Dockerfile para proyecto Nest.js: tutorial práctico

Requerimientos

Instalación

Creando nuestro Dockerfile

Configurando nuestra app

Construir nuestra imagen

Agregando validaciones & Swagger

Docker compose

Recursos

Golang Logging Configuration with Zap: Practical Implementation Tips

Installing the package zap

Setting up the logging

Dockerfile

Reading environment variables

Logger file

Basic setup for projects with Golang

Requirements

Structure project

Dockerfile

go.mod

run.sh

docker-compose.yml

Start app

RELATED

Query large amounts of data from BigQuery faster with Golang

Code without storage api

Storage API

Resources

Geolocation with Postgresql+Nestjs+Typeorm

Publish/Subscribe pattern example (Redis, Kafka)

Explain

docker-compose (Redis)

Publisher (Redis)

Subscriber (Redis)

docker-compose (Kafka)

Publisher (Kafka)

Subscriber (Kafka)

FastApi Example

How does validation work in FastApi?

How to split your React app?

Why do you need to split your application?

Using React.lazy

Split our routes

Kafka introducción & implementación en Nodejs (node+express)

Código del ejemplo

ronnymedina / kafka-nodejs

kafka-nodejs

Tutorial

Using `React.lazy`