DEV Community: Stephanie Bergamo

Go - Traces vs logs, and the four ways I got a trace tree wrong

Stephanie Bergamo — Mon, 27 Jul 2026 16:35:00 +0000

Metrics and logs were the easy part, and I wasn't ready for traces.

It took me a while to get it, first getting a grasp on the vocabulary, then putting it all together and managing to see a tree. I won't blame myself too much, because a trace tree has several ways to fail silently, and I hit them one after another on the same code.

This is the post I wish I'd read first. Not an OpenTelemetry tutorial: the handful of things that stood between me and a working job > phase > task tree, in the order I tripped over them.

Why traces, and not just more logs

Logs and metrics answer questions you decided on in advance. You chose what to log, which counter to bump. They're proactive: you predict what you'll want to know, then instrument for it.

When something you didn't predict goes wrong, you add a log line, redeploy and wait.

Traces flip that. You instrument the structure once (who called what, in what order, nested how) and then you can ask questions you never anticipated, after the fact, without touching the code. "Why was this specific task re-run, and by which worker?" isn't a question you planned for. With a trace, the answer is already sitting in the data, waiting for you to visualize it.

That's the reactive power, and it's why traces are worth the extra pain: they turn "I need to reproduce this with more logging" into "let me go look." The pain is just that all the ways to get it wrong are quiet.

Vocabulary, so the rest makes sense

Five words, because the failures below are mostly about confusing them.

Trace: one tree. Everything sharing a trace_id.
Span: one node, in a tree: a start, an end, some attributes, and a parent.
Span event: a timestamped annotation inside a span. span.AddEvent("reaped"). It lives in the trace.
Wide event: a fat structured log line. Lives in the log system, not the trace. It can carry trace_id and span_id as attributes, which makes it correlatable with a span, but it is not attached to one.
Context: Go's context.Context, carrying the current span. This, and only this, links a child to its parent.

That last point is the whole article. Parent-child comes from the context you pass to Start. Nothing else.

Not events, not naming, not attributes. If the tree is wrong, the context is wrong.

The four traps below aren't random, they follow a span's life cycle: it has to be born (real provider), it has to travel across a process boundary, it has to attach to the right parent, and it has to die so it gets exported. I hit them in exactly that order, and that order is also how you debug them.

Trap 1 - birth: the no-op tracer

otel.Tracer("x") doesn't fail if you call it before the provider is set up. It returns a no-op tracer: it produces spans that are never recorded, never exported, and useless as parents. No panic, no warning...just a silent dud. So if you grab your tracer and start a long-lived span before calling your OTel setup, that span is a ghost, and everything descending from it starts a fresh root instead.

Fix: set up the provider first, before any Start. (Obvious in hindsight; invisible while it's happening, which is kind of the theme here.)

Trap 2 - transport: the silent propagator

Between two processes, context.Context doesn't travel (it's a local in-memory thing). So you serialise the trace context into whatever the transport carries, and rebuild it on the other side: Inject and Extract.

What actually crosses the wire is tiny: a single traceparent string:

00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
|  |                                |                |
|  trace_id (16 bytes)              span_id (8 bytes)|
version                                              flags

That's the whole thing. Version, trace, span, flags.

The trap: Inject needs a propagator to be registered, and if you forget, it writes nothing into the carrier. No warning, no error. Extract on the other side finds nothing and starts a fresh root.

otel.SetTextMapPropagator(propagation.TraceContext{})

One line, once, per binary. Miss it and the wire carries an empty map, silently.

(If you read my Victoria* setup post: same failure mode as the swallowed 404s. The OTel SDK's whole philosophy is that telemetry must never break your app — which is right, and which means it will never tell you it's broken.)

Trap 3 - parent: passing the wrong context

This one's subtle because the tree looks almost right.

I had a job span, a phase span under it, and I was starting each task from the job context:

ctx, span := tracer.Start(c.JobCtx, "task")   // parent = job, skips the phase

One trace, correct-ish parent, no error. But my phase span had zero children — every task hung directly off the job, and the phase sat there empty and pointless. The tree was valid and wrong at the same time.

ctx, span := tracer.Start(c.PhaseCtx, "task")   // parent = phase

One word. The lesson: it's not enough for the context to be a real span context — it has to be the right one. The link is only ever as correct as the context you hand to Start.

(Propagating context across the RPC boundary to the workers is a whole other story, with its own good traps — that one's going in a longer piece.)

Trap 4 - death: the span that never ends

A span is only exported when you call End() on it. A span you never end is never sent — so if it's a parent, its children point at a node that isn't there. A ghost parent, and a tree that looks broken in the UI even though every link is correct.

The task spans were easy: they end when the task completes or when the reaper kills them. But the long-lived ones — job, phase — had no obvious end, so I'd left them open. They never shipped.

Fix: end them deliberately, when the work they represent is actually over.

func (c *Coordinator) Finish() {
    c.phaseSpan.End()
    c.jobSpan.End()
}
// called once Done() is true, before shutting the providers down

And mind the order: end your spans, then flush the providers. The batcher can only ship what's been ended.

Bonus trap: there's no tree view where you'd expect one

Once all four were fixed, my data was correct: one trace, three levels, retried tasks as siblings, and yet no tree in vmui 🤯.
I was surprised, but it's actually expected. VictoriaTraces' built-in UI (/select/vmui), has only Group / Table / JSON modes. It shows spans as flat rows. It does not draw waterfalls.

VictoriaTraces stores and queries traces but delegates visualisation to Jaeger or Grafana, via its Jaeger-compatible API. Point a Grafana Jaeger datasource at http://<host>:10428/select/jaeger/, paste the trace_id, and there's your waterfall. Really, do it, after going through all the trouble, you want a proper visualization.

You can sanity-check the tree without any of that, though: in the Table view, show trace_id and parent_span_id. One trace_id, every task pointing at the phase's span_id. That's the tree, in text.

The pattern

Every one of these failed the same way: no error, wrong tree. No-op tracer, empty propagator, wrong parent context, unended parent: four different bugs, one identical symptom. Telemetry is built to never crash your app, which is great actually, but as a consequence it will never tell you it's misconfigured. The only feedback loop is looking at the data and asking "is this shaped the way I meant?"

That's the real takeaway, more useful than any single fix: when a trace looks wrong, don't debug the tool. Check, in order: did the provider exist yet, is the propagator set, which context did I pass, did the parent end. It's almost always one of those four.

Go - Sending traces, logs and metrics to the Victoria* stack

Stephanie Bergamo — Mon, 27 Jul 2026 16:34:38 +0000

The VictoriaMetrics folks now have a component for each OTel signal: VictoriaMetrics for metrics, VictoriaLogs for logs, and VictoriaTraces for traces. Nice. No collector needed, the Go SDK can talk to all three directly over OTLP/HTTP.

The catch: each one listens on its own port, with its own path, and the paths are almost consistent. Almost.

First, the dependency dance

The OTel Go SDK is split across a lot of modules, and each signal needs its own exporter. I ran go get about fifteen times before everything compiled. Here's the deduplicated version, per component; grab only what you need.

Common to everything:

go get \
  go.opentelemetry.io/otel \
  go.opentelemetry.io/otel/sdk

Traces:

go get \
  go.opentelemetry.io/otel/trace \
  go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp

Metrics:

go get \
  go.opentelemetry.io/otel/sdk/metric \
  go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp

Logs:

go get \
  go.opentelemetry.io/otel/sdk/log \
  go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp

Logs, the logrus way:

go get \
  github.com/sirupsen/logrus \
  go.opentelemetry.io/contrib/bridges/otellogrus

Two things worth knowing:

sdk/metric/metricdata comes with sdk/metric: no separate go get. Same for semconv, which ships with otel itself; you just import the version you want, e.g. go.opentelemetry.io/otel/semconv/v1.41.0.
Watch the spelling on otlptracehttp. I typed otltracehttp first and spent a minute being confused. The pattern is otlp + signal + protocol, consistently, across all three.

The three endpoints

Each signal goes to its own component, on its own port, with its own path:

Metrics → VictoriaMetrics, port 8428 /opentelemetry/v1/metrics
Logs → VictoriaLogs, port 9428 /insert/opentelemetry/v1/logs
Traces → VictoriaTraces, port 10428 /insert/opentelemetry/v1/traces

Spot the odd one out: metrics has no /insert prefix. Logs and traces do. The docs are clear about it, but it's the kind of thing you'll copy-paste wrong at least once (and see the next section for why you won't find out immediately).

Note that this is a local discovery setup, with everything on one box behind raw ports. In real life you'd put a reverse proxy in front and end up with metrics.example.org, logs.example.org, traces.example.org, and save the whole which port for what struggle.

The hidden trap: the SDK swallows your 404s

If you get the path wrong, nothing happens. No error, no panic, no angry log line. The exporter POSTs, gets a 404, and quietly moves on. Your app runs perfectly and your dashboards stay empty.

This is by design (telemetry that crashes your app is worse than no telemetry) but it makes debugging the setup phase miserable. So before anything else, wire up the error handler:

otel.SetErrorHandler(otel.ErrorHandlerFunc(func(err error) {
    log.Printf("otel error: %v", err)
}))

Now you get told. Keep it while setting things up, and honestly, just keep it.

WithEndpoint or WithEndpointURL?

Both exist and they don't take the same thing, which is a fun five minutes.

WithEndpoint takes host:port without a scheme, and needs friends:

otlptracehttp.WithEndpoint("thanos.local:10428"),
otlptracehttp.WithURLPath("/insert/opentelemetry/v1/traces"),
otlptracehttp.WithInsecure(),   // because no scheme, you say it here

Pass http://thanos.local:10428 to WithEndpoint and it breaks — the scheme
isn't part of the value, it's what WithInsecure() is for.

WithEndpointURL takes the whole thing:

otlptracehttp.WithEndpointURL("http://thanos.local:10428/insert/opentelemetry/v1/traces"),

One option instead of three, and http:// already says "insecure", so no
WithInsecure() needed. Use this one.

Both are available on all three exporters (otlptracehttp, otlploghttp,
otlpmetrichttp), with the same behaviour.

Wiring it up

const otelHost = "thanos.local"

Metrics:

metricExporter, err := otlpmetrichttp.New(ctx,
    otlpmetrichttp.WithEndpointURL(
        fmt.Sprintf("http://%s:8428/opentelemetry/v1/metrics", otelHost)),
    otlpmetrichttp.WithTemporalitySelector(
        func(kind sdkmetric.InstrumentKind) metricdata.Temporality {
            return metricdata.DeltaTemporality
        },
    ),
)

That WithTemporalitySelector matters. By default the SDK sends cumulative temporality; delta is what you want here, otherwise your counters get double-counted on restart.

Logs:

exporter, err := otlploghttp.New(ctx,
    otlploghttp.WithEndpointURL(
        fmt.Sprintf("http://%s:9428/insert/opentelemetry/v1/logs", otelHost)),
)

Traces:

exp, err := otlptracehttp.New(ctx,
    otlptracehttp.WithEndpointURL(
        fmt.Sprintf("http://%s:10428/insert/opentelemetry/v1/traces", otelHost)),
)

Share the resource

Build the resource once and hand it to all three providers:

func newResource(service, version string) (*resource.Resource, error) {
    return resource.Merge(
        resource.Default(),
        resource.NewWithAttributes(
            semconv.SchemaURL,
            semconv.ServiceName(service),
            semconv.ServiceVersion(version),
        ),
    )
}

Pro tip: if you build with ldflags, wire it to the same variable you already inject.

If you run several instances of the same service, add semconv.ServiceInstanceID(...) while you're here. Costs one line now, saves a refactor later.

Bonus: logrus straight into VictoriaLogs

If you're on logrus, the contrib bridge means zero changes to your existing log calls:

hook := otellogrus.NewHook(service, otellogrus.WithLoggerProvider(loggerProvider))
logrus.AddHook(hook)

Every logrus.WithField(...).Info(...) now lands in VictoriaLogs as a structured record. Nothing else to do.

Don't forget to shut down

All three providers buffer. Batch processors for logs and traces, a periodic reader for metrics. If your process exits without flushing, you lose whatever was in flight (which, for short-lived processes, can be everything).

Collect the shutdown funcs and call them all:

type Observers struct {
    shutdowns []func(context.Context) error
}

func (o *Observers) Shutdown(ctx context.Context) error {
    var errs error
    for _, fn := range o.shutdowns {
        errs = errors.Join(errs, fn(ctx))
    }
    return errs
}

Call site:

defer func() {
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()
    obs.Shutdown(ctx)
}()

errors.Join rather than an early return, so one failing provider doesn't stop the other two from flushing.

Two things to watch: give Shutdown a fresh context with its own timeout, because, you know, if you pass the one that just got cancelled, the final flush dies before it leaves. And os.Exit() doesn't run deferred functions, so if your code exits that way, your telemetry never ships.

The Victoria* docs are at docs.victoriametrics.com.
Each component has its own OpenTelemetry setup page, which is where the endpoint paths above come from.

Building a B-tree in Go: The gap between theory and implementation

Stephanie Bergamo — Wed, 14 Jan 2026 10:33:08 +0000

I've been obsessed with database internals lately 🤓. It started with Alex Petrov's "Database Internals": one of those books where every page makes you think "wait, THAT'S how it works?!"

But there's a gap between understanding the theory and really getting it. I could explain what a B-tree does to a 6yo, draw the diagrams and the likes. But could I actually build one? That itch needed scratching.

So I dedicated a bit of my time off coding a B-tree from scratch in Go. About 15 hours total, spread across a few days between coffee, sport, rest and debugging sessions. I definitely didn't need one for production (PostgreSQL does this infinitely better), but I wanted to really understand what happens when I type CREATE INDEX.

And honestly? It was one of the most fun coding challenges I've done in a while.

What's a B-tree, anyway?

If you've used a database, you've used a B-tree: they're the data structure behind most database indexes. The key insight: unlike binary trees where each node has 2 children, B-tree nodes can have many children (determined by the "order"). This makes them perfect for disk storage where reading a block is expensive.

I won't explain B-trees from scratch here: Alex Petrov's book does that brilliantly. If you're not familiar with the basics, start with chapter 2 of Database Internals. This article assumes you know what a B-tree is and focuses on what I learned building one.

The Implementation: Easy, Interesting, and "Wait, What?"

The Easy Part: Search

Binary search within nodes, recurse to children. With good tests, this was maybe 2 hours including edge cases. The satisfying kind of easy where you know exactly what to do.

The Interesting Part: Split

Understanding why nodes split (keep tree balanced) vs how to split them (median, two halves, promote) took some head-scratching. Drawing it out on paper helped.

The surprise: root splits are special. You create a brand new root with just the promoted key. The tree grows taller, not wider.

The "Wait, What?" Part: Cascade

Oh wow. This is where I learned the difference between "I understand B-trees" and "I can build a B-tree."

When a split cascades up, you're not just inserting a key: you're managing a whole graph of parent-child relationships. Split an internal node? Better redistribute its children too. And update every child's parent pointer. Miss one? Silent corruption. The debugging session was ...fun...

The full implementation is on GitLab, but the key insight was: when you split an internal node, you're not just moving one or two keys: you're rewiring an entire graph. Get one pointer wrong and your tree silently corrupts. Tests caught this. A lot. And when they didn't and I had to discover that by myself, I wrote new tests to confirm and made sure I fixed the bug.

What Building It Taught Me (That Reading Didn't)

B-tree vs B+tree: Not Just Academic

I knew the theory: B-trees store data everywhere, B+trees only in leaves. But implementing it made me feel the difference.

In my B-tree, search can stop at any level when it finds the key. Elegant! But range scans would be a nightmare (you'd have to traverse the entire tree structure). B+trees sacrifice that early-stop elegance for better range queries (linked leaves).

Now when I see PostgreSQL using B+trees for indexes, I get why, not just what.

Sequential inserts are... weird 🤔

Testing with INSERT 1, 2, 3, 4... created a hilariously unbalanced tree: 7 levels deep with just 50 keys, all growing to the right.

Random inserts? Beautiful 3-level balanced tree.

This is why databases have bulk loading strategies and why REINDEX exists. Seeing my skewed tree made me appreciate those optimizations viscerally.

TDD Isn't Optional (or at least not for me anymore)

I've been building software with TDD for so long now that coding without tests feels... naked? Uncomfortable? Like I'm missing something fundamental.

Could I have built this B-tree without tests? Sure. Would I have shipped a dozen subtle bugs? Absolutely.

Writing tests first isn't about following best practices anymore: it's how I think through implementation. The test tells me what "done" looks like before I start coding.

For this project, that discipline caught:

Duplicate key insertion (oops, forgot to check)
Index out of bounds on edge cases (classic off-by-one)
Children count mismatch after splits (n keys → n+1 children, remembered that in the test)
Parent pointers pointing to the wrong parent (caught by checking parent relationships in tests)

When cascade finally worked and all 20+ tests turned green simultaneously? That's the confidence TDD gives you. I wasn't hoping it worked: I knew it did. That's the payoff: shipping with certainty, not crossed fingers. (by now I guess you might have understand that I'm a strong believer in the increased productivity and overall quality TDD is bringing).

And yes, 100% coverage throughout. Some call it obsessive. I call it professional 😇.

The "It Just Clicked" Moment

There was this moment, probably 10 hours in, debugging why searches failed after cascade. I was staring at my tree visualization (can't help but thanks Copilot for that helper function!), and suddenly I saw it: the children weren't following their parent after the split.

That moment when the entire structure suddenly made sense (not just conceptually, but in memory, pointers, the whole graph of relationships) that's what hands-on gets you. No amount of reading gives you that.

The Numbers

Time: ~15 hours across 4 days
Code: ~200 lines of implementation
Tests: 20+ unit tests, 100% coverage
Bugs found: Too many to count (but TDD caught them all)
Coffee consumed: Significant

What's Next

This was way more fun than I expected. There's something deeply satisfying about implementing a data structure you've used for years without really understanding it.

Next up: LSM-trees (Log-Structured Merge trees). They represent the opposite tradeoff: B-trees optimize for reads (in-place updates), LSM-trees optimize for writes (append-only). Understanding both will show me why Cassandra chose LSM while PostgreSQL chose B+trees.

Reading about compaction strategies is one thing. Implementing them will be another level entirely 🤩.

If you're curious about database internals, start with "Database Internals" by Alex Petrov. Then pick something and build it. The gap between reading and implementing is where the real ~magic~ learning happens.

You can see the full implementation on my self-hosted GitLab (yes, I run my own instance: infra nerd life). Pull requests welcome if you spot ways to improve it! 😊

Hugo.io - Multiline cells in a table

Stephanie Bergamo — Fri, 10 Jun 2022 07:22:26 +0000

A problem

Building table in markdown is super easy and well-documented.

However, having a bit of formatting within it, like carriage return within a cell as below is not possible by default.

Hugo uses Goldmark (a CommonMark implementation in Go) to process the markdown. Apparently it's extremely fast, with this implementation we choose to block any HTML processing d'HTML by default.

A solution

Once you know this, solving this is pretty straightforward.

You can customize plenty of Goldmark options in the configuration file (config.toml/yaml/json).
So what we want to do is autorising Goldmark to render HTML content :

[markup.goldmark]
    [markup.goldmark.renderer]
    unsafe = true

From now, within the table, we just need to add <br> tag as below :

|Adresse | Parking de La Poste<br> Le Bourg <br>97122 Baie-Mahault |

That's it !

Free tips

If you don't want to bother with | when building tables, someone very nice has built a table generator.

We thank this person, from the bottom of our heart 💜.

Testing with Kotlin and JUnit5

Stephanie Bergamo — Fri, 10 Jun 2022 07:10:53 +0000

The problem

I've recently started coding backend and mobile stuff in kotlin, following a book, then starting a pet project, ...

In the meantime, I've also started to gain a lot of interest on Test Driven Development and found myself wondering "wow, how do I test in Kotlin ? which framework should I use?"

To learn further, I've deciced to start a serie of articles on this topic:

Basic testing in Kotlin with JUnit5 from the Java ecosystem (this article)
Basic testing in Kotlin with Kotest built for kotlin specificatlly (upcoming)
Mocking, stubbing and contract testing in Kotlin with JUnit5 and Kotest (upcoming)
A comparision of these two testing frameworks (mostly in term of features, usability, readability), this one might be a bit opinionated. (upcoming)

I'm a java-ist so my go-to test framework is JUnit, this is the sole reason why I'm starting by this one.

Agenda

Getting started
Execution and lifecycle
A very simple test
Checking exceptions
The power of parameterized tests
Conditional tests
Conclusion

If you just want to jump in the code, please yourself

Basics on testing with JUnit5

Setup

For the Gradle users, here's the setup.

To be able to use the JUnit5 features, we must first add the dependencies:

  testImplementation("org.junit.jupiter:junit-jupiter-api:5.8.2")
  testImplementation("org.junit.jupiter:junit-jupiter-engine:5.8.2")

Then we must specify we want to use JUnit to run the tests :

// For Gradle + Groovy 
tasks.test {
    useJUnitPlatform()
}
// For Gradle + Kotlin 
tasks.test {
   useJUnitPlatform()
}

The entire necessary configuration can be found here.

Note: I'm using gradle 7.3.3.

Tests execution and orchestration

Junit will consider as a test any function annotated with @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, or @TestTemplate.

We also have annotation to help us wrap test execution:

@BeforeAll executed first, before the whole test suite. useful for instiantiating external dependencies for instance.
@BeforeEach executed after BeforeAll and before any test, useful when we need to ensure the state is clean before launching for example.
@AfterEach not surprisingly, executed after any test.
@AfterAll executed at the end of the test suite, for housekeeping purpose, pushing stats or whatever.

A very simple test

Now we're all set, time to code and test!

Let's say we have a useless class like this one :

class Dollar(val value: Int) {
    var amount: Int = 10

    operator fun times(multiplier: Int) {
        amount = value.times(multiplier)
    }
}

Obviously we want to test this class has the expected behavior, with a very basic test:

We'll need to first import the JUnit5 helpers :

import org.junit.jupiter.api.Assertions
import org.junit.jupiter.api.Test

And then we can create our test as follows:

@Test
fun `should multiply correctly`() {
    val five = Dollar(5)
    five.times(2)
    Assertions.assertEquals(10, five.amount)
}

We can now execute this, using our favorite IDE or the simple gradle command :

./gradlew test

A quick word on naming

Well, no strong rule here, but a test method should :

test one thing, and one thing only
have an explicit name so the expectation is clear

You may have notice that I'm using backtick here.

This is a Kotlin capability, identifier for variable and method can use them, and although there's nothing mandatory here, I find it clearer.

What about checking exceptions ?

Ok, now we don't want to multiply our dollar by zero, so we change our code a bit, to raise an exception if this occurs.

Let's write the test first:

@Test
fun `should throw exception when multiplying by 0`(){
    val one = Dollar(1)
    assertThrows<NoMoneyException> {
        one.times(0)
    }
}

Yep, it's just that easy, okay, this does not even compile, since the NoMoneyException class does not exists. Let's create it !

class NoMoneyException(message: String?) : Throwable(message) {
}

We then update our times operator :

operator fun times(multiplier: Int) : Dollar {
    if (multiplier == 0) {
        throw NoMoneyException("Can't multiply by zero")
    }
    return Dollar(amount.times(multiplier))
}

You can run since and see the green test :)

Let's execute the same test with different inputs !

I think you'll agree, if we add more test cases, we'll be loosing readability and we'll duplicate the same test again and again.

Grouped assertions

Well there's a few great Kotlin assertions that comes with Junit, let's play with assertAll and a collection with a multiplier and the expected result.

@Test
fun `should multiply using stream`() {
    val five = Dollar(5)
    val inputs = arrayListOf(
        arrayListOf(2, 10),
        arrayListOf(3, 15),
        arrayListOf(10, 50)
    )
    assertAll(
        "should provide the expected result",
        inputs
            .stream() // Remove this line and use the collection directly
            .map {
                { assertEquals(Dollar(it[1]).amount, five.times(it[0]).amount) }
            }
    )
}

Parameterized tests

Well, I've used a lot of Table Driven Tests in golang, this is super helpful to write compact and repeatable tests.

With JUnit, we can achieve the same with Parameterized tests.

Parameterized tests makes tests more readable and avoid duplicates, but don't take my word for it, let's code !

First, we'll need to add a new dependency :

   testImplementation("org.junit.jupiter:junit-jupiter-params:5.8.2")

Now let's replace our previous example and use a CsvSource with the multiplier and the expected value :

@ParameterizedTest(name = "multiply {0} by 5 should return {1}")
@CsvSource(
    "2, 10",
    "3, 15",
    "10, 50",
)
fun `should multiply correctly`(multiplier: Int, expected: Int ){
    val five = Dollar(5)
    Assertions.assertEquals(Dollar(expected).amount, five.times(multiplier).amount)
}

So basically, we :

add the @ParameterizedTest annotation and optionnally define a name,
add the annotation for the the type of argument source we want to provide (@CsvSource here) with the related test cases
enjoy !

You may have noticed the name of the test? Well, this little trick makes our test super explicit and easier to debug, see for yourself :

With that, you can directly see which testcase fails, see for yourself:

I don't know about you, but I personnally find it more readable than the group assertions.

For more info about customizing the display name, see this part of the JUnit documentation.

Another nice thing, is that we can use several types of arguments as inputs:

ValueSource allows to pass a list of arguments of primitive types, String and Class: useful for a testing a single argument. See an example here
CsvSource as show-cased here, we can pass an unlimited list of arguments as a string representing a CSV input
CsvFileSource same as the previous file, except we use a CSV file. See an example here
EnumSource lets you run the same test for each constant (or a selected set) of a given ENUM. See an example here
MethodSource this one is super powerful, we can basically have anything we want as input source (say a JSON or a Parquet file), process it with the MethodSource and use it to execute our tests. See an example here
ArgumentSource this one goes even further than MethodSource. With a new class, implementing the ArgumentProvider interface, you can generate input data. See an example here

We also have a bit of syntaxic sugar with @NullSource, @EmptySource and @NullAndEmptySource.

Be careful when using BeforeEach and "parameterized" or "repeated" tests

Each item of a parameterized or repeated test suite is considered as a single test.

Therefore, whatever is defined in BeforeEach will be executed before each occurence of the test source.

Say we define a BeforeEach as follows :

@BeforeEach
fun initEach() {
    println("hello I'm a test")
}

After executing should multiply correctly which have 3 rows in its CsvSource, we'll have the following output:

hello I'm a test
hello I'm a test
hello I'm a test

Do we really want to execute this test ?

JUnit comes with multiple way to decide whether a test sould be executed or not, depending of the context.

First we can totally disable a test or a class, by adding the @Disabled annotation.

In addition, we can programmatically define condition execution depending on:

the Operating System, using @EnabledOnOs, @DisabledOnOs and the OS Enum:

@EnabledOnOs(OS.MAC)
@EnabledOnOs(OS.MAC, OS.LINUX)

the JRE, with @EnabledOnJre, @EnabledForJreRange, @DisabledOnJre, @DisabledForJreRange and the JRE Enum:

@EnabledOnJre(JAVA_8)
@EnabledForJreRange(min = JRE.JAVA_11, max = JRE.JAVA_16)

system properties, with @EnabledIfSystemProperty and @DisabledIfSystemProperty

@EnabledIfSystemProperty(named = "os.arch", matches = ".*64.*")

one or multiple environment variable(s), @EnabledIfEnvironmentVariable, @EnabledIfEnvironmentVariables, @DisabledIfEnvironmentVariable and @DisabledIfEnvironmentVariable:

@EnabledIfEnvironmentVariable(named = "EXEC_ENV", matches = ".*ci.*")
@EnabledIfEnvironmentVariable(named = "DEBUG", matches = "enabled")

Note that the two singular annotations are repeatable.

a custom condition*, using @EnabledIf and @DisabledIf with a method name or its FQN (if the method is not in the same class) as string:

@EnabledIf("execAlways")
@Test
fun `something to test`(){}

private fun execAlways(): Boolean {
    return true
}

Conclusion

That's it for this first part, covering the basics on testing Kotlin code with JUnit, we can already check a lot of things with :

grouped assertions
exception check
parameterized tests
conditional tests

I hope you'll find this useful, you can find the full test implementation here.

Ingesting an S3 file into an RDS PostgreSQL table

Stephanie Bergamo — Fri, 10 Jun 2022 07:02:30 +0000

The problem

I'm using Redshift for a while now, and one feature I find particularly useful, is the ability to load a table from the content of an S3 file:

COPY table_name
FROM 's3://bucket-name/path/file.ext'
  iam_role 'arn:aws:iam::<aws-account-id>:role/<role-name>'
  region 'region-code';

Lately, I needed to do the very same thing with a PostgreSQL database, hosted on RDS.

The good news is : YES WE CAN !

The solution

Setup

AWS pre-installs (among other things) 2 extensions on our RDS :

aws_commons
aws_s3

Now, since we need to interact with S3, we simply need to run the following command, assuming
our user is a superuser or has database owner privileges:

CREATE EXTENSION aws_s3 CASCADE;

This command installs both aws_commons and aws_s3.

Okay, that was the easy part.

Well, now we have to work on the necessary IAM role for our RDS instance...and that's where we'll find the first pitfall...

Under the hood, installing the aws_s3 extension also adds an IAM role, managed by AWS (and therefore, read only for us)
to interact with plenty of AWS services.
As we can see below, the role is linked to the S3Import function :

The problem is, we can only link a single role to a function and this one does not have any access to our S3 bucket, therefore this cannot work.

In the end, even if the policy is super short in the doc, here is what we actually need :

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "s3import",
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::bucket-name",
                "arn:aws:s3:::bucket-name/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "rds:CrossRegionCommunication"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:AllocateAddress",
                "ec2:AssociateAddress",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:CreateNetworkInterface",
                "ec2:CreateSecurityGroup",
                "ec2:DeleteNetworkInterface",
                "ec2:DeleteSecurityGroup",
                "ec2:DescribeAddresses",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeCoipPools",
                "ec2:DescribeInternetGateways",
                "ec2:DescribeLocalGatewayRouteTables",
                "ec2:DescribeLocalGatewayRouteTableVpcAssociations",
                "ec2:DescribeLocalGateways",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets",
                "ec2:DescribeVpcAttribute",
                "ec2:DescribeVpcs",
                "ec2:DisassociateAddress",
                "ec2:ModifyNetworkInterfaceAttribute",
                "ec2:ModifyVpcEndpoint",
                "ec2:ReleaseAddress",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:CreateVpcEndpoint",
                "ec2:DescribeVpcEndpoints",
                "ec2:DeleteVpcEndpoints",
                "ec2:AssignPrivateIpAddresses",
                "ec2:UnassignPrivateIpAddresses"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "sns:Publish"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup"
            ],
            "Resource": [
                "arn:aws:logs:*:*:log-group:/aws/rds/*",
                "arn:aws:logs:*:*:log-group:/aws/docdb/*",
                "arn:aws:logs:*:*:log-group:/aws/neptune/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "logs:DescribeLogStreams"
            ],
            "Resource": [
                "arn:aws:logs:*:*:log-group:/aws/rds/*:log-stream:*",
                "arn:aws:logs:*:*:log-group:/aws/docdb/*:log-stream:*",
                "arn:aws:logs:*:*:log-group:/aws/neptune/*:log-stream:*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kinesis:CreateStream",
                "kinesis:PutRecord",
                "kinesis:PutRecords",
                "kinesis:DescribeStream",
                "kinesis:SplitShard",
                "kinesis:MergeShards",
                "kinesis:DeleteStream",
                "kinesis:UpdateShardCount"
            ],
            "Resource": [
                "arn:aws:kinesis:*:*:stream/aws-rds-das-*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:PutMetricData"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "cloudwatch:namespace": "AWS/RDS"
                }
            }
        }
    ]
}

Once created, we we'll need to :

unlike the managed role from the s3Import function;
link the new role to the s3Import function;

Careful now, swapping and applying the role may takes a few minutes, don't believe what the AWS console tells you.

That's it now, we're all set !

Usage

To load the content of a CSV file from S3, we'll just need to :

SELECT aws_s3.table_import_from_s3 (
   'target-table',
   'columns list, separated by a comma, OPTIONAL',
   'DELIMITER '','' CSV HEADER',
     aws_commons.create_s3_uri(
           'bucket-name',
           'file-prefix',
           'region-code'
       )
    );

When providing an empty string as column list, the command will try to map the file columns to the table columns.

The pitfalls

Constraints

Here are some constraints to keep in mind, not necessarily documented :

the bucket name must not contain a period . (I got this error message : S3 bucket names with a period (.) are not supported). Yeah I know we should have path-style S3 bucket, but hey, one has to deal with legacy right ?!
; cannot be used as a delimiter, but just so you know, this is not a PostgreSQL limitation as it accepts any character as a delimiter, as long as it's single one-byte character;
just like the COPYcommand from PostgreSQL only text, CSV, or binary format are supported;

Error handling

When an error occurs, the PG client doesn't get a real, explicit error message. I mean, either for an unsupported bucket
name or a missing access right, we get the very same error message :
Unable to generate pre-signed url, look at engine log for details

We must go dig in the PG logs, from the RDS console to get an explicit error message and debug efficiently.

Conclusion

Once we get through the installation and understand the limits and pitfalls, this feature is super useful and easy to use.

Unfortunately (well, this is not really surprising) we can't install the two extensions cannot on simple EC2.
Life would be so nice if we could use AWS feature as we wish, where we wish !

So we get to choose:

either we go for RDS, but we stick to the AWS handpicked extensions (exit timescale, citus or their columnar storage, ... ),
or we use other solutions, and we deprive ourselves of all the benefits of hosted / managed services.

Originally published at https://g33k.life/en/tech/load-rds-via-s3/ on December 16, 2020.