DEV Community

Augusts Bautra
Augusts Bautra

Posted on

How do you deal with test record leaks?

I've written about test leaks before, they are a pernicious, hard to debug thing that must be avoided at all costs.

Today I got a weird failure on CI and while investigating, it became apparent that some other spec is leaving records in the test DB, working outside DB transaction.

I know of two common ways to sidestep transactionality:

  1. Creating records in a before(:all) block. Sometimes this may be necessary to save on repeat setup time, but look into TestProf's before_all helper for a safer approach.
  2. Creating records in inline code outside example context:
describe "#some_method" do
  # the two users fabricated in this hash will leak.
  {
    regular: FactoryBot.create(:user), 
    elevated: FactoryBot.create(:super_user),
  }.each_pair do |permission_level, record| 
    it "works for '#{#{permission_level}}'" do
      # some asserts ...
    end
  end
end
Enter fullscreen mode Exit fullscreen mode

To avoid cases like this in the future, I recommend setting up a simple hook that checks the test DB has no records of popular models used in your app.
Place in a file location that gets loaded by spec helper, usually spec/support/hooks/:

class RecordsLeftInTestDBError < StandardError; end

RSpec.configure do |config|
  config.after(:suite) do
    counts = {
      some_records: SomeModel.count,      
    }

    counts.values.sum.positive? &&
      # I'd prefer `abort` to `raise`, but for some reason
      # it  does not fail the RSpec run when called from an
      # after-suite hook like this.
      raise(
        RecordsLeftInTestDBError.new(
          "Leak detected!\n" \
          "Records in the test DB after running the suite:\n" \
          "#{counts}",
        ),
      )
  end
end
Enter fullscreen mode Exit fullscreen mode

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more

Top comments (0)

AWS Security LIVE!

Tune in for AWS Security LIVE!

Join AWS Security LIVE! for expert insights and actionable tips to protect your organization and keep security teams prepared.

Learn More