DEV Community: Youngjoon Lee

Exploring rust-libp2p

Youngjoon Lee — Wed, 10 May 2023 04:43:39 +0000

Connectivity Tests

In libp2p.io, the current state of Transport implementations in each programming language is already written. But, I’ve seen there have been so many changes in rust-libp2p, as described in the rust-libp2p in 2022 blog post.

After reading the libp2p Connectivity document, I've tested if 'dialing' works for each of the following scenarios using rust-libp2p v0.51.3.

Scenario	Dialing	Transports tested
Standalone -> Standalone	Successful	TCP, WebSocket, WebRTC¹
WASM browser -> Standalone	Successful	WebSocket²
WASM browser <- Standalone	Failed³	WebSocket
WASM browser -> WASM browser	Failed³	WebSocket
JS browser⁴ -> Standalone	Successful	WebRTC
Private -> (Relay⁵) -> Private	Successful	TCP
Private⁶ -> (Relay⁵) -> Private with Hole-punching	Failed⁷	TCP

WASM Limitations

I've found that the following features of rust-libp2p cannot be enabled for WASM.

gossipsub, mdns, dns, tokio

If WASM codes import those features, the following error occurs:

error[E0432]: unresolved import `libp2p::gossipsub`
  --> core/src/p2p.rs:20:5
   |
20 |     gossipsub, identity,
   |     ^^^^^^^^^ no `gossipsub` in the root

In regard to gossipsub especially, the gossipsub has been disabled for the wasm32-unknown-unknown target by rust-libp2p/pull/2506#issuecomment-1036448620
because of the following reasons:

The custom Interval implementation in rust-libp2p wasn't performant: rust-libp2p/issues/2497. So, they decided to revert back to wasm-timer by rust-libp2p/pull/2506.

However, the rust-libp2p with wasm-timer cannot be compiled for wasm32-unknown-unknown with the following error:

$ wasm-pack build

[INFO]: 🎯  Checking for the Wasm target...
[INFO]: 🌀  Compiling to Wasm...
   Compiling libp2p-gossipsub v0.45.0 (https://github.com/libp2p/rust-libp2p.git?branch=master#14938043)
error[E0599]: no method named `checked_add` found for struct `wasm_timer::Instant` in the current scope
   --> /Users/yjlee/.cargo/git/checkouts/rust-libp2p-98135dbcf5b63918/1493804/protocols/gossipsub/src/peer_score.rs:872:34
    |
871 |   ...                   let window_time = validated_time
    |  _________________________________________-
872 | | ...                       .checked_add(topic_params.mesh_message_deliveries_window)
    | |                           -^^^^^^^^^^^ method not found in `Instant`
    | |___________________________|
    |

rust

Fortunately, it seems that we can use the instant crate instead of wasm_timer::Instant. This try had been done by rust-libp2p/pull/2320, but has been reverted by rust-libp2p/pull/2506 as above.
But the one remaining problem was that the gossipsub needs the Interval struct which is also dependant on the Instant. To resolve this, the libp2p team is thinking of contributing an Interval implementation to the futures-timer crate: rust-libp2p/issues/2497#issuecomment-1038923700
But, I think the easier solution would be just implementing the missing wasm_imter::Instant.checked_add() function above into the wasm_timer because the wasm_timer already provides the Interval struct which is being used by rust-libp2p in production. However, it seems that the libp2p the doesn't want to keep maintaining the wasm-timer crate.

WebRTC example ↩
vincev/wasm-p2p-chat based on vincev/libp2p-websys-transport that will be included in the rust-libp2p offically ↩
Browser nodes cannot listen for incoming conns. ↩
js-libp2p-webrtc browser-to-server example ↩
Relay-server example ↩
dCUtR example ↩
It seems that the hole-punching is not always successful. Used AWS for relayer and listener (in private subnet with NAT), and my laptop for dialer. ↩

Why prevote in Tendermint

Youngjoon Lee — Wed, 10 May 2023 04:41:53 +0000

Tendermint Byzantine Fault Tolerance (hereafter BFT) consensus algorithm has two stages of voting before committing a block to the state,
while Raft consensus algorithm (that doesn't cover the Byzantine problem) has a single stage of preparation (aka. log replication) for committing a transaction.
When I first met Tendermint, I was most curious why Tendermint has the two-stage voting (prevote-precommit).

In Ethan Buchman 2016, he mentioned as below:

In asynchronous environments with Byzantine validators, a single stage
of voting, where each validator casts only one vote, is not sufficient to ensure
safety. In essence, because validators can act fraudulently, and because there
are no guarantees on message delivery time, a rogue validator can co-ordinate
some validators to commit a value while others, having not seen the commit,
go to a new round, within which they commit a different value.

A single stage of voting allows validators to tell each other what they
know about the proposal. But to tolerate Byzantine faults (which amounts,
essentially to lies, fraud, deceit, etc.), they must also tell each other what
they know about what other validators have professed to know about the
proposal. In other words, a second stage ensures that enough validators
witnessed the result of the first stage.

That sounds reasonable, but I wanted to see a specific counterexample that explains why only precommit without prevote is not sufficient.

Fortunately, I found a counterexample from a Reddit post.
Below, I am going to go into details on this counterexample.

Let's assume that Tendermint has only single stage of voting (aka. precommit), and the network consists of the following validator nodes:

100 validator nodes in the network
- 28 offline nodes (e.g. due to network partitioning)
- 65 honest nodes
- 1 honest node (aka. H)
- 1 Byzantine node (aka. B)

Let's say the current round is n. A "valid" block is proposed by one of the validators.
Then, all 66 (= 65 + 1) honest nodes broadcast a precommit for this valid block.
But, let's imagine the case when the Byzantine node B sends nil to 65 honest nodes and itself, but a precommit to the honest node H.

In this case, H has 67 precommits (> 100 * 2/3) with its own vote. Then, H will commit the block.
However, all other nodes (65 honest + B) move to the next round n+1 because they have only 66 precommits (< 100 * 2/3).

This result is a breach of safety because the honest node H committed the block that all other honest nodes didn't commit.

If Tendermint has the prevote-precommit mechanism, the above example can be resolved safely.
Let's replace all precommits in the above example with prevotes.
Then, H will broadcast a precommit to all other nodes instead of committing the block to its state,
but it won't receive 2/3+ precommit from other nodes because all other nodes except H won't broadcast a precommit since they didn't receive 2/3+ prevotes.
As a result, all nodes will move to the next round n+1 without committing the block.
Then, the consensus remains safe, even if any new block never be committed due to B in subsequent rounds.