DEV Community

Aaroophan Varatharajan
Aaroophan Varatharajan

Posted on Originally published at aaroophan.Medium on

When COMMIT Stops Being a Local Decision: Rethinking Distributed Computing #13

On one database, commit is a decision. Across several machines, commit becomes an agreement that has to survive failures.

How Two-Phase Commit (2PC) coordinates distributed transactions across multiple machines, handling prepare, commit, failures, crash recovery, blocking, and atomicity.
"How Two-Phase Commit (2PC) coordinates distributed transactions across multiple machines, handling prepare, commit, failures, crash recovery, blocking, and atomicity."

Consider a transaction on one database.

Change some data.

If everything succeeds: COMMIT

If something goes wrong: ROLLBACK

The database owns the relevant state, so there is one ✨obvious✨ place where the final decision happens.

Atomicity feels pleasantly local:

Either all of this transaction becomes permanent, or none of it does.

Now spread the data across multiple servers.

Account A lives on Server 1.

Account B lives on Server 2.

We want one transaction to:

  • A = A + 1
  • B = B - 1

Logically, this is still one indivisible transaction.

Physically, two different machines have to modify two different pieces of data.

And that tiny change ruins the simplicity of COMMIT.

Because Server 1 can succeed while Server 2 fails.

The transaction is logically one thing.

The machines performing it are not.


Partial Success Is Worse Than Failure

Suppose a transaction coordinator tells:

  • Server 1 → add 1 to A
  • Server 2 → subtract 1 from B

Both operations execute tentatively.

Now imagine the simplest possible protocol:

Once you’ve done your part, commit it.

Server 1 commits.

Server 2 crashes before committing.

We now have:

  • A = A + 1
  • B = unchanged

The system did not merely fail to complete the transaction.

It completed half of it permanently.

That is much worse.

If neither side had committed, we could simply tell the client:

The transaction failed. Try again.

The previous consistent state would still exist.

But once one participant commits and another does not, retrying is no longer obviously safe. Some of the intended change already escaped into permanent state.

The uncomfortable realization is:

Distributed atomicity is not about making every operation succeed. It is about preventing different participants from reaching incompatible final outcomes.

Oracle describes distributed transactions in exactly this all-or-nothing form: changes span multiple databases, but the transaction must still commit everywhere or roll back everywhere as one unit.

So perhaps the coordinator should make the decision for everybody.


Fine. Broadcast COMMIT

The coordinator waits until both servers have performed their local work.

Then it sends:

  • COMMIT → Server 1
  • COMMIT → Server 2

Much cleaner.

One authority.

One decision.

Both participants do what they are told.

Except the network gets a vote.

Server 1 receives: COMMIT

Server 2 does not.

Maybe its machine crashed.

Maybe the message disappeared.

Maybe the connection failed.

Server 1 commits.

Server 2 does not.

We are back where we started.

The coordinator decided correctly.

That was not enough.

The hidden problem is that sending the final decision does not prove that every participant is capable of carrying it out.

So before the system says:

Commit.

it needs another question.

Can you commit?

That extra question is where Two-Phase Commit begins.


The First Phase Is Not COMMIT

In Two-Phase Commit , or 2PC , one process acts as the transaction coordinator and the machines containing pieces of the transaction act as participants.

The coordinator does not immediately order everyone to commit.

It first sends something like: PREPARE

Meaning:

If I later tell you to commit this transaction, are you able to guarantee that you can do it?

Each participant checks its local situation.

Did its part of the transaction execute successfully?

Can constraints still be satisfied?

Can it preserve the information required to finish later?

Can it keep the necessary transaction state?

If not, it votes: NO

If it can guarantee completion, it votes: YES

Oracle’s description of the prepare phase captures the important semantic difference: a prepared participant has not yet committed, but it promises that it will later be able to follow the global commit or rollback decision even across an intervening failure.

This is much stronger than:

Everything looks fine right now.

A YES vote is a promise about the future.

And promises in distributed systems need receipts.


“YES” Has to Survive a Crash

Imagine Server 1 replies: YES

Then immediately crashes.

When it restarts, what should happen?

If the promise existed only in RAM, the server wakes up with no idea that it had agreed to participate in an unfinished distributed transaction.

That makes the YES meaningless.

So before a participant says it is prepared, it must preserve enough transaction state in stable storage.

PostgreSQL’s PREPARE TRANSACTION illustrates this principle directly: once prepared, the transaction's state is stored on disk so it can later be committed or rolled back even if a database crash occurs before the final decision.

Oracle similarly describes prepared participants recording recovery information in redo logs and retaining the resources needed to complete the transaction after failure.

So the participant’s logic becomes roughly:

  • perform tentative work
  • write prepared state durably
  • vote YES

Not:

  • vote YES
  • hope nothing happens

The durable log is what turns:

I think I can commit.

into:

I can recover into a state from which I can still honor this decision.


Prepared Means You Have Given Up Some Freedom

This is the part that makes the first phase more important than its name suggests.

Before voting YES , a participant can still say:

Something went wrong. Abort my local work.

After voting YES , it has promised not to make that decision independently.

It is now prepared.

It holds enough state to commit.

It may also retain locks or other resources required to preserve correctness.

And then it waits.

The participant cannot decide:

This is taking too long. I’ll just roll back.

Because the coordinator may already have received YES votes from everyone and decided to commit globally.

Oracle explicitly describes a prepared node as waiting for the global COMMIT or ROLLBACK decision rather than making a unilateral choice. The prepared transaction can remain in-doubt until that decision becomes known.

That is the trade.

The participant gained durability of intent.

It lost local authority over the outcome.


Now the Coordinator Can Actually Decide

Suppose the coordinator asks three participants: PREPARE?

and receives:

  • P1 → YES
  • P2 → YES
  • P3 → YES

Only now does it know something it did not know in our naive protocol:

Every participant has reached a state from which it can complete the transaction.

So the coordinator can choose: GLOBAL COMMIT

and send the decision to everybody.

But suppose the votes are:

  • P1 → YES
  • P2 → NO
  • P3 → YES

Then commit is impossible.

The only safe global outcome is: GLOBAL ABORT

Likewise, if a participant cannot prepare and the coordinator eventually treats that participant as failed, the transaction can be aborted before the global commit decision.

This gives us the two phases:

Phase 1: Prepare

  • Coordinator → PREPARE
  • Participants → YES / NO

Phase 2: Decision

If everyone voted YES : Coordinator → COMMIT

Otherwise: Coordinator → ABORT

This is why the protocol needs two phases.

The first establishes whether commitment is possible everywhere.

The second establishes which outcome everyone must perform.

One phase cannot safely answer both questions after the system has been distributed.


Before the Decision, Aborting Is Cheap

Suppose the coordinator sends PREPARE to three participants.

Two reply: YES

The third never responds.

Maybe it crashed.

Maybe communication failed.

At this point the coordinator has not decided to commit.

That matters.

Because no irreversible global commitment has been made, the coordinator can decide: ABORT

The participants that were prepared can roll back their tentative work.

The transaction disappears.

We return to the previous consistent state.

This is one of the nice properties of the prepare boundary:

Before universal readiness exists, failure can still collapse safely into abort.

No participant is allowed to commit merely because its own local work succeeded.

Local success is not global success.

That distinction is exactly what the naive protocol was missing.


Then We Cross the Uncomfortable Line

Now take the other case.

Every participant votes: YES

The coordinator has enough information to decide: COMMIT

Suppose it sends that decision to Server 1.

Server 1 commits.

Then the network fails before Server 2 receives the message.

We cannot now say:

Never mind. Abort everything.

Server 1 has already made the committed state permanent.

The global decision has crossed a boundary.

The transaction must now be driven toward commit everywhere.

This creates what the lecture calls the critical window: after the system has committed to the global outcome but before every participant has successfully completed and acknowledged it.

The coordinator cannot safely pretend the transaction never happened.

The remaining participants cannot independently guess the answer.

The system has entered the most awkward state distributed transactions produce:

The outcome exists, but not everybody knows it yet.


Crash Recovery Becomes Part of COMMIT

Suppose Server 2 crashed after voting YES but before receiving the final decision.

When it comes back, its durable log says: Transaction 42 → PREPARED

That tells Server 2:

I promised I could finish this transaction, but I do not know whether the global outcome was COMMIT or ABORT.

So it must recover the prepared transaction and determine the coordinator’s decision.

If the decision was COMMIT , it commits.

If it was ABORT , it rolls back.

The prepared state is precisely what allows this recovery to happen without guessing.

Similarly, the coordinator must durably remember its own decision.

Imagine it decides COMMIT , sends the message to one participant, then crashes.

When it restarts, it cannot wake up and reconsider:

Maybe let’s abort instead.

Somebody may already have committed.

So the coordinator’s global outcome also belongs in stable storage.

MIT’s treatment of 2PC makes this durability requirement explicit: the coordinator force-writes its decision, prepared workers recover into the prepared state and ask for the outcome, and the coordinator retains the outcome until workers acknowledge it.

The log is no longer merely an optimization for crash recovery.

It is part of the protocol’s memory.


Then the Coordinator Disappears at Exactly the Wrong Time

Now we reach the part where 2PC stops looking like a magical atomicity machine.

A participant has voted: YES

It is prepared.

It has preserved its state.

It is waiting for: COMMIT or ABORT

Then the coordinator becomes unreachable.

What should the participant do?

It cannot simply commit.

Perhaps another participant voted NO and the coordinator chose abort.

It cannot simply abort.

Perhaps everyone voted YES , the coordinator chose commit, and another participant already committed.

So the prepared participant waits.

Its uncertainty is legitimate.

Oracle calls this an in-doubt distributed transaction : failure during 2PC can leave prepared participants waiting for the outcome, with associated data remaining locked until the transaction can be resolved.

Academic treatments describe the same property more bluntly:

Two-Phase Commit is a blocking protocol.

If all surviving participants are prepared but none knows the global decision, they may have no safe action except waiting for the failed coordinator or another source of decisive information to recover.

That is not an implementation bug.

It follows from what *YES * means.


The Prepared Participant Is Stuck Because It Is Being Correct

This state can feel ridiculous.

The server is alive.

Its database is alive.

Its disks are alive.

It has the transaction data.

And yet it refuses to either commit or roll back.

Why?

Because it lacks authority.

Before preparing, the participant owned its local choice.

After preparing, it delegated the final choice to the distributed commit decision.

Now the coordinator is unavailable.

The participant is not waiting because it is incapable of performing either operation.

It is waiting because it cannot determine which operation preserves atomicity.

That is an important distinction.

The server is technically capable.

The system is informationally stuck.

The Origami Software Engineer has seen this recurring distributed-systems pattern in another costume:

Missing communication becomes missing knowledge.

And once the missing knowledge determines whether permanent state should exist, guessing becomes dangerous.


Locks Make the Waiting Visible

Prepared transactions cannot casually release everything they were using.

If another transaction could modify the same state while the original transaction’s outcome remained unresolved, the participant might no longer be able to honor its promise correctly.

So prepared transactions may retain locks.

Now uncertainty starts affecting unrelated work.

Transaction 42 is in doubt.

Transaction 73 wants the same data.

Transaction 73 waits.

Oracle’s documentation explicitly notes that in-doubt distributed transactions can keep data locked until the outcome is resolved; its error documentation even describes resources remaining locked by prepared distributed transactions.

This is where availability pays the price for atomicity.

2PC’s promise is:

We will not let half the distributed transaction permanently win while the other half permanently loses.

The cost is that, under some failures, the system may prefer to wait rather than guess.

Correctness is being protective.

Possibly overprotective.

But for good reason.


Two-Phase Commit and Two-Phase Locking Are Solving Different Problems

The names are similar enough to cause unnecessary suffering.

Two-Phase Locking is about concurrent transactions.

It helps preserve consistency and isolation by controlling the ordering of conflicting accesses.

Two-Phase Commit is about one transaction whose work spans multiple participants.

It helps preserve atomicity and durability by coordinating whether every participant commits or every participant aborts.

One asks:

Can these concurrent operations safely interleave?

The other asks:

Can these distributed pieces make one final decision?

A system may need both.

Locks can ensure that other transactions do not corrupt the objects involved.

2PC can ensure that the distributed transaction does not commit on only some machines.

Different problem.

Different mechanism.

Same uncomfortable theme:

Once state is distributed, a property that looked local requires coordination.


The Extra Phase Is Buying Certainty

2PC is not cheap.

Every distributed transaction now involves more messages.

Before commit:

  • PREPARE
  • votes

Then:

  • COMMIT / ABORT
  • acknowledgements

Prepared state has to be written durably.

The coordinator must persist enough information to recover its decision.

Participants may retain locks while waiting.

Failures can leave transactions in doubt.

That can reduce availability.

The transaction that once ended with: COMMIT has become a small distributed protocol.

And that is precisely the point.

The additional machinery exists because the simple operation stopped carrying enough information.

One local database can know:

My work succeeded.

A distributed transaction needs to know:

Everyone can complete the same outcome, and that outcome will survive crashes and communication failures.

That stronger claim costs more.


COMMIT Became a Conversation Because Nobody Owned the Whole Transaction

At the beginning, the problem looked tiny.

Move one unit:

  • A = A + 1
  • B = B - 1

When both values lived under one transactional authority, atomicity could be enforced locally.

Then A and B moved onto different machines.

Now Server 1 could succeed while Server 2 failed.

So independent commit was unsafe.

We introduced a coordinator.

But one COMMIT broadcast could still reach only some participants.

So the coordinator first asked whether everybody was prepared.

A YES vote became a durable promise.

Once everybody promised, the coordinator could choose one global outcome.

Then failures created another problem: some participants might know the decision while others remained uncertain.

So participants and coordinator had to preserve protocol state across crashes.

And once a prepared participant could no longer discover the decision, waiting became safer than guessing.

That is why COMMIT stops being one instruction in a distributed transaction.

It becomes a conversation about readiness, authority, durability, and shared outcome.

Two-Phase Commit does not make failures disappear.

It creates a disciplined boundary around what the system is allowed to do when failures happen.

Before the boundary, abort is still safe.

After the boundary, the chosen outcome has to be carried through.

The extra phase was never ceremony.

It was the price of turning several independent machines back into one atomic decision.

That’s not failure.

That’s evolution.


The “I liked this” Starter Pack:

Don’t let your fingers get lazy now.

  • Like : It tells me this was worth writing.
  • A Comment: Tell me your thoughts, your favorite snack, or a better title for this blog.
  • Boost it: Especially with that one developer who definitely needs this.

Thanks for being here. It genuinely helps more than you know!

— Aaroophan Varatharajan

Find me elsewhere:

Top comments (0)