DEV Community

loading...

Fix Your dbt Concurrent Transaction Database Error

Corissa (She/Her)
Data QA Engineer focused on investigating data quality, and how data can be tested with automation in useful ways.
・1 min read

found on Unsplash

If you use Fishtown Analytics' data build tool (dbt) for data engineering and development, you may have increased your project's thread count. This is easily done in the Profiles.yml file under your .dbt/ folder for various reasons, from running dbt test faster to configuring dbt projects for multithreading.

The Concurrent Transaction Error

But sometimes, increasing your thread count can lead to errors like this common database error showing a concurrent transaction conflict.

Completed with 1 error and 0 warnings:

Database Error in model test_model (models/stg/test_model.sql)
  could not complete because of conflict with concurrent transaction
  compiled SQL at target/compiled/of_models/stg/test_model_column_name.sql
Enter fullscreen mode Exit fullscreen mode

A Potential Quick Fix

This fix doesn't work for every case, but if
a) you're not trying to multithread and
b) if you're getting this error and can't even run your models, it's a quick fix.

Try returning to a single thread for dbt run. Then you can increase your thread count when you're ready later.

That's your quick fix for the dbt database error for running concurrent transactions. Happy programming!

Discussion (0)