DEV Community

Ramanan
Ramanan

Posted on

bulk_destroyer: destroy_all that fires every callback, without one DELETE per row

#rails #ruby #activerecord #performance

A Rails gem that destroys records and their dependent: :destroy trees with set-based SQL while still running every destroy callback, including the children's.

delete_all is fast but skips callbacks. destroy_all runs every callback but loads each record and deletes it with its own query, and does the same for every child. On a tree with a few thousand parents and tens of thousands of children, that is tens of thousands of queries.

I wanted both: every before_destroy, around_destroy and after_destroy hook, on every record in the tree, with bulk SQL underneath. So I built bulk_destroyer.

Post.where(author_id: 42).bulk_destroy_all
post.bulk_destroy!
Enter fullscreen mode Exit fullscreen mode

What you get

Per batch, it issues one SELECT per association per level and one DELETE per table per level. Every record's destroy callbacks still run, children's included.

Benchmark (Rails 8.1.3.1, same-machine database, single runs; each post has 5 comments plus likes and other dependents):

Database Posts / rows destroy_all bulk_destroy_all Queries Speedup
SQLite 1,000 / 17,003 9.44s 1.34s 32,001 → 45 7.1x
SQLite 5,000 / 85,003 58.33s 6.69s 160,001 → 221 8.7x
PostgreSQL 16 1,000 / 17,003 19.73s 1.58s 32,001 → 45 12.5x

The hook calls are identical: 84,000 for the 1,000-post case in both. The savings are in queries, not in skipped work. With a network between your app and the database the gap should be larger, since every removed query is a removed round trip.

The tricky part: after_destroy

Rails runs a record's callbacks as one call, with the DELETE in the middle:

run_callbacks(:destroy) do
  # the DELETE happens here
end
Enter fullscreen mode Exit fullscreen mode

before_* runs before the block and after_* runs after it. In a bulk destroy there is no per-record DELETE, so each record's chain has to be split: run the before hooks, wait while one DELETE removes everyone, then run the after hooks.

Plain Ruby can't suspend a method in the middle, but a Fiber can. Each record's run_callbacks(:destroy) runs in its own Fiber, and the block passed to it calls Fiber.yield:

  1. start: resume each Fiber. Before hooks run, then it pauses.
  2. bulk DELETE: one statement per table covers the level while the Fibers wait.
  3. finish: resume each Fiber. After hooks run, now genuinely after the row is gone.

This is not concurrency. Only one Fiber runs at a time and nothing is parallel. Fibers are used as a pause button so Rails' own callback machinery (ordering, conditions, throw :abort, around_destroy) runs unmodified.

What is supported

  • dependent: :destroy, :delete_all, :nullify, :restrict_with_error, :restrict_with_exception
  • polymorphic associations
  • has_many :through (join rows only, like Rails) and HABTM
  • belongs_to ..., dependent: :destroy / :delete
  • counter caches
  • commit callbacks (after_commit on: :destroy)

Each batch runs in a transaction(requires_new: true): a real transaction on its own, a savepoint inside yours. If a destroy callback raises ActiveRecord::Rollback, which Rails' transaction would otherwise swallow silently, you get ActiveRecord::RecordNotDestroyed instead of a quiet skip.

Limits worth knowing

  • Fiber ceiling. Each record holds one paused Fiber until its level is deleted. Ruby on Linux fails at roughly 31,700 live Fibers (two memory mappings per Fiber and the default vm.max_map_count). Batches are therefore capped by record count (max_batch_records, default 10,000) and split in half recursively when a tree is bigger. A single root whose own tree exceeds the cap raises BulkDestroyer::TreeTooLarge before anything runs, unless you set oversized_tree = :inline.
  • Splitting isn't atomic across parts. Each part is its own transaction. Wrap the call in your own transaction if you need all-or-nothing.
  • hook_mode = :inline skips Fibers entirely, but then after_destroy runs before the DELETE rather than after.
  • dependent: :destroy_async is not supported.
  • Rails 7.0 is not supported. Tested on Rails 7.1 through 8.2 (alpha) with SQLite and PostgreSQL; the code uses portable ActiveRecord APIs and adapter-aware IN-list limits, so other adapters should work, but I haven't tested them.

Install 

gem "bulk_destroyer"
Enter fullscreen mode Exit fullscreen mode

Source, README and benchmarks: https://github.com/rneelakantan/bulk_destroyer. Issues and PRs welcome, especially results from MySQL/MariaDB and other Ruby versions.

Top comments (0)