How to execute promises using concurrency, advantages and pitfalls

Ismael Santana — Wed, 28 Aug 2024 00:00:00 +0000

Hi folks! Recently, one of my friends showed me a piece of code that was taking about 12 seconds to execute. The code looked something like the example below. I want to discuss some small improvements we can make, along with their advantages and potential pitfalls.

import { setTimeout } from "timers/promises";

async function mockIOCall({ delayInMs = 1000 }) {
  await setTimeout(delayInMs);
  if (mockError) throw mockError;
}

async function main() {
  const someIOCallResult = await mockIOCall({ delayInMs: 9000 });
  const anotherIOCallResult = await mockIOCall({ delayInMs: 3000 });
}

In the code above, it's just a regular JavaScript code to be executed by the NodeJS Runtime nothing inherently "wrong" with it. However, this code could run faster by executing both promises "simultaneously".

Huh? 🤔

Yes, I know, this was my first reaction when I was learning about this as well. Let’s say the first promise, doSomeIOCall, takes 9 seconds to complete, and the second promise, doAnotherIOCall, takes 3 seconds. In the original code, the total time is 12 seconds. However, with a small refactor, we can improve this code so that it takes only 9 seconds, which is the time taken by the longest IO operation in this example.

How can we do it?

//...rest of the code

async function main() {
  const [someIOCallResult, anotherIOCallResult] = await Promise.all([
    mockIOCall({ delayInMs: 9000 }),
    mockIOCall({ delayInMs: 3000 }),
  ]);
}

The Promise.all accepts a list of promises, and return an list of the value returned by every promise ordered by the registration order of the list of input

That's it this seed end here... And this is something that I would like to say.

But we have a SERIOUS PROBLEM with this approach...

The Problem with `Promise.all`

The Promise.all follows "fail fast" principle. This means that if any of the promises in the Promise.all call fails, the entire operation will be canceled, throwing an error that you'll need to handle it.

So when this happens at beginning and no promises has been fulfilled, this is amazing, because it's stops early, but let's imagine the worst-case scenario: what happens to the promises that have already completed their tasks when one of the promises fails?

I will give to you a few seconds to think about it...

Correct! They remain completed, but you can't perform any cleanup or rollback functions for them after a failure. Let's modify our mockIOCall signature to illustrate this:

async function mockIOCall({ delayInMs = 1000, mockError = null }) {
  await setTimeout(delayInMs);
  if (mockError) throw mockError;
}

Now, the code below will throw an error after 9 seconds, but the promise that takes 3 seconds to complete will have already done its job.

async function main() {
  const [someIOCallResult, anotherIOCallResult] = await Promise.all([
    mockIOCall({
      delayInMs: 9000,
      mockError: new Error("Some Annoying Error"),
    }),
    mockIOCall({ delayInMs: 3000 }),
  ]);
}

What If We Use try-catch?

Well, by doing this, you can handle the error, but it really depends on your specific case and what you want to do with the exception. If your desired behavior is "If something goes wrong, that's okay; we don't need to rollback the successful promises," then this is a good solution.

async function main() {
  try {
    const [someIOCallResult, anotherIOCallResult] = await Promise.all([
      mockIOCall({
        delayInMs: 9000,
        mockError: new Error("Some Annoying Error"),
      }),
      mockIOCall({ delayInMs: 3000 }),
    ]);
  } catch (exception) {
    console.error(exception.message);
  }
}

But for scenarios where the "all or none" approach isn't acceptable and we need to handle each error individually, Promise.all isn't the best solution. Instead, we should use Promise.allSettled, which is very similar to Promise.all, but it returns an array of results for all promises, whether they were fulfilled or rejected.

import { setTimeout } from "timers/promises";

async function mockIOCall({ delayInMs = 1000, mockError = null }) {
  await setTimeout(delayInMs);
  if (mockError) throw mockError;
}

async function main() {
  const [someIOCallResult, anotherIOCallResult] = await Promise.allSettled([
    mockIOCall({
      delayInMs: 5000,
      mockError: new Error("Some Annoying Error"),
    }),
    mockIOCall({ delayInMs: 3000 }),
  ]);
  if (someIOCallResult.reason) {
    console.error(someIOCallResult.reason.message);
    // do anything necessary
  }
  if (anotherIOCallResult.reason) {
    console.error(someIOCallResult.reason.message);
    // do anything necessary
  }
}

References and you must read it, trust me

Promise Concurrency

Promise.all In MDN Docs

Promise.allSettled In MDN Docs

📘 Seed

For now, this is all I have to write about it. When I notice something that can be improved in this seed, I will update it. But if you find something that could be added, or if I said something incorrect, feel free to correct me in the comments, and I'll update the seed later. Thank you for your time reading up to here!

Gerenciando Buckets no S3 usando a AWS CLI

Ismael Santana — Sun, 20 Mar 2022 01:05:22 +0000

Para isso assumo que você o AWS CLI já esteja configurando com um IAM só com permissões para o S3.

Listando buckets

Para listar buckets da sua conta amazon é bastante similar ao ls de sistema Unix, sendo a sintaxe assim: aws s3 ls

Listando conteúdo dentro de um bucket

Ponto importante o nome do bucket deve começar com o s3://*nome
do bucket*

aws cli ls s3://i3s-01

Criando Buckets

Para criar buckets utilizamos a seguinte sintaxe: aws s3 mb s3://teste-123
Para facilitar o entendimento é bom associar o mb com Make Bucket.

Como podemos ver um novo bucket foi criado, e logo conseguimos ver o mesmo com o comando ls

Deletando Buckets Vázios

Bem similar ao comando de criação, usamos os aws s3 rb s3://teste-123
Nesse caso para fins didaticos associe o rb com Remove Bucket

Deletando bucket com objetos dentro

Caso você tente remover um bucket com objetos dentro é retornado o seguinte erro:

Para resolvermos esse problema adicionamos a flag --force

aws s3 rb s3://i3s-01 --force

Buckets públicos para leitura no AWS S3

Ismael Santana — Sat, 19 Mar 2022 23:35:15 +0000

Um dos jeitos de resolver o acesso a documentos de buckets no S3 é configurando o ACL, para aceitar ações de em qualquer objeto no bucket, assim evitando erros como esse:

Solução

Um ponto importante, essa solução cria um ACL que deixa o o bucket inteiro como público, porém podemos configurar com que apenas pasta especifica, ou objetos tenham esse acesso público de leitura

Nas nossas configurações de ACLs colocamos o seguinte JSON

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowPublicRead",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::i3s-01/*"
        }
    ]
}

Com essa configuração estamos dizendo que qualquer tentativa de leitura de arquivos recurso i3s-01/ é pública, um ponto de atenção é que devemos especificar o caminho do bucket sendo arn:aws:s3:::i3s-01/*

Para isso também devemos desmarcar os seguintes campos na seção Bloquear Acesso Público

Resultado

Assim configuramos esse bucket como livre para leitura:

DEV Community: Ismael Santana