DEV Community

Erick Correa
Erick Correa

Posted on

Getting a NestJS app's logs into Elasticsearch without writing grok

My team is rolling out ELK observability across four services, and I drew the short straw of instrumenting the first one: a NestJS API. I expected an afternoon of tedium. I got that, plus one surprise that made the whole exercise worth writing up.

The starting point

The app had exactly three log statements in 1,100 lines of TypeScript:

// main.ts
bootstrap().catch((err: unknown) => {
  new Logger('Bootstrap').error(
    'Failed to start application',
    err instanceof Error ? err.stack : String(err),
  );
  process.exit(1);
});

// key-value.service.ts
console.error(err);
console.log(updateKeyValueDto);
Enter fullscreen mode Exit fullscreen mode

That last line dumps an entire request DTO into the logs. Whatever a client sends, raw, in production output. We had never noticed because nobody reads stdout on a service that mostly works.

The plan was standard: structured JSON logs in ECS format (Elastic Common Schema), shipped by Filebeat, so Kibana gets real fields instead of text to grep. If you emit ECS from the app, you skip the part everyone hates: no grok patterns, no ingest pipelines, no custom index mappings. The app speaks the schema and Filebeat just forwards it.

The surprise: our only error log was dead code

To verify the setup I booted the app with the database down, expecting to see that Bootstrap error in the new format.

Nothing. The process printed a raw error object and exited.

It turns out NestFactory.create() has abortOnError: true by default. When a module fails to initialize, which is what happens when TypeORM can't reach Postgres, Nest logs the error itself and calls process.exit(). The promise never rejects, so bootstrap().catch(...) never runs. Our one production error log only fired for failures that happen after the app object exists, like a busy port. A database outage at startup, the most likely failure in real life, bypassed it completely.

The fix is one option:

const app = await NestFactory.create(AppModule, {
  bufferLogs: true,
  abortOnError: false, // create() now throws, so the catch actually catches
});
Enter fullscreen mode Exit fullscreen mode

Check your own NestJS services. If your bootstrap catch assumes it sees every startup failure, it probably doesn't.

The retrofit

NestJS with no dedicated logger means the lightest path is pino. Elastic maintains an ECS formatter for it, and nestjs-pino replaces the Nest logger app-wide, including the HTTP request logs you get for free:

// app.module.ts
LoggerModule.forRoot({
  pinoHttp: {
    ...ecsFormat({ convertReqRes: true }),
    base: { 'service.name': process.env.SERVICE_NAME ?? 'fulfillment-api' },
    level: process.env.LOG_LEVEL ?? 'info',
  },
}),
Enter fullscreen mode Exit fullscreen mode

The bare console.error(err) became a log with context and the ECS error trio:

this.logger.error(
  { err, 'event.action': 'key-value.create', 'event.outcome': 'failure' },
  'key-value create failed',
);
Enter fullscreen mode Exit fullscreen mode

And the DTO-dumping console.log was deleted. Some log calls should be improved; that one just needed to die.

Here is the bootstrap failure after the change, captured from a real run with the database down:

{"log.level":"error","@timestamp":"2026-09-15T17:45:33.275Z","ecs.version":"8.10.0",
 "event.action":"app.bootstrap","event.outcome":"failure","service.name":"fulfillment-api",
 "error":{"type":"Error","message":"connect ECONNREFUSED 127.0.0.1:59999",
 "stack_trace":"Error: connect ECONNREFUSED..."},"message":"failed to start application"}
Enter fullscreen mode Exit fullscreen mode

Every field is queryable. In Kibana you can now alert on event.outcome: failure, group errors by event.action, and read stack traces from a field instead of reassembling multiline text.

Shipping it

Because the app emits ECS JSON, the Filebeat config is short and generic:

filebeat.inputs:
  - type: container
    paths:
      - /var/lib/docker/containers/*/*.log
    parsers:
      - ndjson:
          target: ''
          overwrite_keys: true
          add_error_key: true

output.elasticsearch:
  hosts: ['${ELASTIC_HOST}']
Enter fullscreen mode Exit fullscreen mode

That's the entire server-side story. The ndjson parser promotes the app's fields and Filebeat's index template already understands ECS.

One rough edge to know about: with convertReqRes: true, the response side converts cleanly to http.response.*, but the request object kept pino-http's raw req shape in my version. The logs are still valid and queryable, just not fully ECS on the request side. Worth checking your formatter version if you need strict http.request.* fields.

The one-command version

I did this retrofit with a Claude Code skill I wrote, which runs the whole process: it inventories every log call, shows a keep/enrich/drop table for approval before touching anything (that inventory is what caught the DTO leak), wires the right ECS formatter for whatever logger the project already uses, rewrites the calls, and generates the Filebeat config. The run on this app is where every example in this post came from.

I packaged it, with the real output above included as examples, here: https://erickcmdev.gumroad.com/l/ecs-logging ($9).

If you'd rather do it by hand, everything you need is in this post. Either way, boot your app with the database down and watch what your bootstrap catch actually does. Mine never fired once.

This is my first post, really appreciate the time you nerds invested reading this :)

Top comments (0)