DEV Community: SoYounJeong

Substrate Pallet Code Analysis: Sudo

SoYounJeong — Thu, 28 Jul 2022 05:12:00 +0000

Overview

In substrate-based chain, there is a god-like account who can call a privileged function which is called sudo account. Sudo pallet composed of that function what sudo account can do. There should be only one sudo account for the Runtime, which is called 'sudo-key'. Before launching public blockchain network, having sudo account could be useful but after launching, most of the Parachain projects tend to get rid of this Sudo pallet for safety.

Let's look at how this pallet is implemented.

Source Code Analysis

Storage

There is only one storage for this pallet. Storage type is StorageValue to store only one sudo key account for Runtime.

#[pallet::storage]
#[pallet::getter(fn key)]
pub(super) type Key<T: Config> = StorageValue<_, T::AccountId, OptionQuery>;

Dispatch Call

sudo

Dispatch a call with Root origin.

Params

origin: who calls this function

call: Some Call type like dispatch calls. When building runtime, Compiler collects all the Call type into one single Call Enum type by #[pallet::call] macro. Code below is how Substrate deals with Call data type.

In Runtime,

pub enum Call {

    System( <Pallet<Runtime> as Callable<Runtime>>::Call ),
    Balances( <Pallet<Runtime> as Callable<Runtime>>::Call ),
    Staking( <Pallet<Runtime> as Callable<Runtime>>::Call ),
    ...
}

For example, in Staking Pallet, any dispatch call is defined in Call enum type

pub enum Call {
   ...
   Staking(pallet_staking::Call::<Runtime>)
   ...
}

pub enum Call {
    bond { controller, value, payee },
    nominate { targets }
    ....
}

Analysis

Check whether origin is signed.
Check whether caller is 'sudo-key account'
dispatch_bypass_filter: Skip checking origin filter in dispatch call.
Emit 'Sudid' event
Sudo account doesn't pay fee

pub fn sudo(
    origin: OriginFor<T>,
    call: Box<<T as Config>::Call>,
) -> DispatchResultWithPostInfo {

    // 1
    let sender = ensure_signed(origin)?;

    // 2
    ensure!(Self::key().map_or(false, |k| sender == k), Error::<T>::RequireSudo);

    // 3 
    let res = call.dispatch_bypass_filter(frame_system::RawOrigin::Root.into());

    // 4
    Self::deposit_event(Event::Sudid { sudo_result: res.map(|_| ()).map_err(|e| e.error) });

    // 5 
    Ok(Pays::No.into())
}

sudo_unchecked_weight

The logic of this method is same with _sudo but this time, Sudo account can control the Weight of the call. As we can see the parameter we give as _weight is used in the weight calculation in #[pallet::weight((*_weight, call.get_dispatch_info().class))] macro._

Params

_weight: Amount of weight sudo want to control

#[pallet::weight((*_weight, call.get_dispatch_info().class))]
pub fn sudo_unchecked_weight(
    origin: OriginFor<T>,
    call: Box<<T as Config>::Call>,
    _weight: Weight,
) -> DispatchResultWithPostInfo {

    // Same logic with sudo
}

set_key

Set sudo key with different account

Params

new: New sudo key account. Type is Source type of StaticLookup trait. This type is for converting any source type into AccountId. Let's take a closer look!

First of all, StaticLookup is defined as

type Source: any type that we want to look
type Target: Type that we want to covert
fn lookup: Convert Source type into Target type
fn unlookup: Convert Target type into Source type

pub trait StaticLookup {
    // 1
    pub type Source;
    // 2 
    pub type Target;
    // 3 
    fn lookup(s: Self::Source) -> Result<Self::Target, LookupError>
    // 4
    fn unlookup(t: Self::Target) -> Self::Source;
}

In frame system pallet, Target is defined with AccountId type, which means any Source type that we want to look can be converted into AccountId type when we call lookup method.

pub trait Config {
    ...

    type lookup: StaticLookup<Target = AccountId>

    ...
}

In Runtime, when we look at how frame_system's config is implemented for Runtime, pallet_indices is defined as lookup, which means pallet_indices must implement StaticLookup trait.

impl frame_system::Config for Runtime {

    ...

    type lookup = Indices(pallet_indices)

    ...
}

Inside pallet_indices

impl StaticLookup for Pallet
MultiAddress: Type of address formats
fn lookup_address(): Returns address if exists depending on the formats of address. Here, type of address format we are looking for is Account Id and AccountIndex.

// 1
impl<T: Config> StaticLookup for Pallet<T> {
    type Source = MultiAddress<T::AccountId, T::AccountIndex>,
    type Target = T::AccountId,

    fn lookup(a: Self::Source) -> Result<Self::Target, LookupError> {
        Self::lookup_address(a).ok_or(LookupError);
    }

    fn unlookup(a: Self::Target) -> Self::Source {
       Multiaddress::Id(a)
    } 
}

// 2
pub enum MultiAddress<AccountId, AccountIndex> {
    Id(AccountId), // public address
    Index(AccountIndex), // Account index for database
    ...
}

// 3
impl<T: Config> Pallet<T> {

    fn lookup_index(index: T::AccountIndex) -> Option<T::AccountId> {
        // where Accounts is a storage that maps _AccountIndex_ to (AccountId, Balance)
        Accounts::<T>::get(index).map(|x| x.0)

    }  

    fn lookup_address(a: Multiaddress<T::AccountId, T::AccountIndex> -> Option<T::AccountId> {
         match a {
              Multiaddress::Id(account) => Some(account)
              Multiaddress::Index(index) => Self::lookup_index(index)
         }
    }
}

#[pallet::storage]
pub type Accounts<T: Config> = StorageMap<_, Blake2_128Concat, T::AccountIndex, (T::AccountId, BalanceOf<T>, bool)>;

Analysis

Check whether origin is signed.
Check whether caller is 'sudo-key account'
Convert MultiAddress type into AccountId,
Emit 'KeyChanged' event
Put new sudo-key into Key storage
Sudo doesn't pay a fee

#[pallet::weight(0)]
pub fn set_key(
    origin: OriginFor<T>,
    new: <T::Lookup as StaticLookup>::Source,
) -> DispatchResultWithPostInfo {

    // 1
    let sender = ensure_signed(origin)?;

    // 2
    ensure!(Self::key().map_or(false, |k| sender == k), Error::<T>::RequireSudo);

    // 3
    let new = T::Lookup::lookup(new)?;

    // 4
    Self::deposit_event(Event::KeyChanged { old_sudoer: Key::<T>::get() });

    // 5
    Key::<T>::put(&new);

    // 6
    Ok(Pays::No.into())
}

sudo_as

Sudo account calls and dispatches a call with a signed account.
Make signed account as sudo account which pays no fee
Params

who: The one who calls the dispatch call

pub fn sudo_as(
    origin: OriginFor<T>,
    who: <T::Lookup as StaticLookup>::Source,
    call: Box<<T as Config>::Call>,
) -> DispatchResultWithPostInfo {

    let sender = ensure_signed(origin)?;
    ensure!(Self::key().map_or(false, |k| sender == k), Error::<T>::RequireSudo);

    let who = T::Lookup::lookup(who)?;

    let res = call.dispatch_bypass_filter(frame_system::RawOrigin::Signed(who).into());

    Self::deposit_event(Event::SudoAsDone {
                sudo_result: res.map(|_| ()).map_err(|e| e.error),
            });

    Ok(Pays::No.into())
}

Conclusion

Sudo is special feature that only exists in Substrate based chain which can do everything and it is not preferred to have in public blockchain.

Feel free to leave a comment if you have some questions.

Reference

Sudo
Frame System
Indices

Polkadot Consensus Part 1: BABE Algorithm Analysis

SoYounJeong — Fri, 08 Jul 2022 02:19:06 +0000

Why do we need consensus?

Every blockchain system has its own consensus algorithm. It is the way that the nodes in a decentralized network are able to stay synced with each other. It is the process by which these nodes communicate and come to agreement, and are able to produce new blocks. To secure the blockchain network, it is crucial to consistently produce the blocks. It should never stop. Today, let's closely look at consensus algorithm of Polkadot, which is BABE.

Polkadot's Consensus

Babe Overview

One of the consensus protocol Polkadot/Kusama use is BABE(Blind Assignment for Blockchain Extension). It is a slot-based block production mechanism which uses a VRF function to randomly perform the slot allocation for validators(authorities).

Polkadot's block time can be split into epoch and slot. Epochs can be broken into slots and every slot is six seconds long.

On every slot, all validators(authorities) generate a random number from VRF and if it is lower than threshold value which is proportional to weight/amount of stake(Currently, amount of stake has nothing to do with threshold value) they have right to produce new block. Since it is fully randomized and probabilistic way, which Polkadot called Primary Slots, there are some chances that no validators(authorities) are chosen for specific slot. To prevent not to be delayed for producing blocks, Polkadot has made Secondary slots, which is also using VRF but deterministic way. Moreover, there could be multiple authorities for single slot, which could lead to temporal fork. BABE's fork-choice rule is weight-based, which mean chains that has more primary slots are chosen for the system.

Code Overview

So let's look at how slot-based BABE is implemented.

claim_slot

Explanation

Tries to claim the given slot.

Params

slot: pub struct Slot(u64)
epoch: pub struct Epoch { some fields }
keystore: Pointer to keystore. Arc

Returns

Option<(PreDigest, AuthorityId)>
PreDigest: Enum type that contains all data required to validate a block such as Primary, Secondary, SecondaryVRF
AuthorityId: Public address of authority


// Get authorities for specific epoch
// authorities = Vec[(authorityId, index), ...]
// pub struct Slot(u64);
// epoch.authorities = [(authority's public address, weight)]

pub fn claim_slot(
    slot: Slot,
    epoch: &Epoch,
    keystore: &SyncCryptoStorePtr,
) -> Option<(PreDigest, AuthorityId)> {

    let authorities = epoch
        .authorities
        .iter()
        .enumerate()
        .map(|(index, a)| (a.0.clone(), index))
        .collect::<Vec<_>>();

    claim_slot_using_keys(slot, epoch, keystore, &authorities)
}

claim_slot_using_keys

Explanation

Claim Primary and claim Secondary

Params

slot,epoch,keystore are mentioned above
keys: [(author's public address, index for epoch)]

Returns

Option<(PreDigest, AuthorityId)>

Comment

In claim_primary_slot, there is epoch.config.c. This is pre-defined constant when we build our node. That is, pub const PRIMARY_PROBABILITY: (u64, u64) = (1, 4)

// claim primary. If fail, claim secondary
pub fn claim_slot_using_keys(
    slot: Slot,
    epoch: &Epoch,
    keystore: &SyncCryptoStorePtr,
    keys: &[(AuthorityId, usize)],
) -> Option<(PreDigest, AuthorityId)> {

    claim_primary_slot(slot, epoch, epoch.config.c, keystore, keys).or_else(|| {
        if epoch.config.allowed_slots.is_secondary_plain_slots_allowed() ||
            epoch.config.allowed_slots.is_secondary_vrf_slots_allowed()
        {
            claim_secondary_slot(
                slot,
                epoch,
                keys,
                keystore,
                epoch.config.allowed_slots.is_secondary_vrf_slots_allowed(),
            )
        } else {
            None
        }
    })
}

claim_primary_slot

Explanation

If the result of VRF, here, it is inout, is less than threshold returns Some((pre_digest, authority_id.clone() else returns None.

fn claim_primary_slot(
    slot: Slot,
    epoch: &Epoch,
    c: (u64, u64),
    keystore: &SyncCryptoStorePtr,
    keys: &[(AuthorityId, usize)],
) -> Option<(PreDigest, AuthorityId)> {

    ---snippet---

    let threshold = calculate_primary_threshold(c, authorities, *authority_index);
    if check_primary_threshold(&inout, threshold) {
    let pre_digest =
PreDigest::Primary(PrimaryPreDigest {
        slot,
        vrf_output: VRFOutput(signature.output),
        vrf_proof: VRFProof(signature.proof),
        authority_index: *authority_index as u32,
    });
    return Some((pre_digest,
authority_id.clone()))}

    None
}

Secondary Slot

Explanation

If there is no primary authorities, set up to back-up plan.
sencondary_author should return authority if there is no primary slot authority.

Params

slot,epoch,keys,keystore are same with above.
author_secondary_vrf: bool type for whether VRF secondary slots are allowed.

Returns

Option<(PreDigest, AuthorityId)>

fn claim_secondary_slot(
    slot: Slot,
    epoch: &Epoch,
    keys: &[(AuthorityId, usize)],
    keystore: &SyncCryptoStorePtr,
    author_secondary_vrf: bool,
) -> Option<(PreDigest, AuthorityId)> {

    if authorities.is_empty() {
    // It means there is authorities for primary slot. Return Immediately
    return None
    }

    let expected_author = secondary_slot_author(slot, authorities, *randomness)?;

    --snippet--
}

pub(super) fn secondary_slot_author(
    slot: Slot,
    authorities: &[(AuthorityId, BabeAuthorityWeight)],
    randomness: [u8; 32],
) -> Option<&AuthorityId> {
    if authorities.is_empty() {
        return None
    }

    let rand = U256::from((randomness, slot).using_encoded(blake2_256));

    let authorities_len = U256::from(authorities.len());
    let idx = rand % authorities_len;

    let expected_author = authorities.get(idx.as_u32() as usize).expect(
        "authorities not empty; index constrained to list length; \
                this is a valid index; qed",
    );

    Some(&expected_author.0)
}

BabeApi

Explanation

After claim primary/secondary slot, it is used in BabeAPi for give epochs.
Result would be used for fork-choice rule.

Variables

claims: Hashmap
EpochAuthorship: Struct{ primary: Vec, Secondary: Vec, SecondaryVRF: Vec }

async fn epoch_authorship(&self) -> RpcResult<HashMap<AuthorityId, EpochAuthorship>> {

    --snippet--

    for slot in *epoch_start..*epoch_end {
            if let Some((claim, key)) =
                authorship::claim_slot_using_keys(slot.into(), &epoch, &self.keystore, &keys)
            {
                match claim {
                    PreDigest::Primary { .. } => {
                        claims.entry(key).or_default().primary.push(slot);
                    },
                    PreDigest::SecondaryPlain { .. } => {
                        claims.entry(key).or_default().secondary.push(slot);
                    },
                    PreDigest::SecondaryVRF { .. } => {
                        claims.entry(key).or_default().secondary_vrf.push(slot.into());
                    },
                };
            }

    Ok(claims)
}

Further Thought

When BABE calculates threshold value in *calculate_primary_threshold_ , they have used the sum of all authorities' weight which is equal to 1 as denominator. It doesn't seem like threshold value is proportional to (weight/stake) as they said. Since validators' weight in Polkadot/Kusama is same, amount of stake is not important for current BABE system.

Reference

Polkadot Consensus Part 3: BABE. This is part 3 in our Polkadot… | by Joe Petrowski | Polkadot Network | Medium

Joe Petrowski ・ Dec 19, 2019 ・
Medium

Polkadot's Consensus Protocols · Polkadot Wiki

The Consensus Mechanisms of Polkadot.

wiki.polkadot.network

Polkadot NPoS Overview

SoYounJeong — Sat, 04 Jun 2022 18:47:27 +0000

Polkadot is the fully-sharded blockchain that is built to connect and secure the other blockchains as layer-0.

Polkadot has interesting feature that it gives any Dot holders a role to be part of network security entities, called Nominated-Proof-of-Staking(NPoS).

So what is NPoS?

Overview

What is NPoS?

To continuously produce blocks and secure the network in blockchain, it is important to have good validators. In normal proof-of-staking, validators are the entities who have the great amount of stakes in the network, which can lead to be centralized network. However, in Polkadot, the system encourages any Dot holders to participate in securing the network in two ways, Nominators and Validators. In general, most of the crypto users would be a nominator because being a validator requires lots of works. Nominators can select up to 16 validators who they trust with minimum bonded asset. Then, a limit number of validators(not all validators!) would be elected by the system who have the most backing Dot. Both locked their assets as collateral and receive staking rewards. Most importantly, staking system pays out staking rewards equally to all elected Validators not proportional to their amount of stake. The network creates an incentive to lower-staked validators so when certain time passed, Polkadot system wants to achieve as much equally-staked validator set as possible.

Let's say there are two validation pools, A with 1000 Dot and B with 500 Dot and rewards are 2000 Dot for each. Then, someone who backed 200 Dot in pool B will get more rewards than someone who backed 200 Dot in pool A

Finally, rewards are distributed into nominators with proportional to their amount of stake.

NPoS Scheme

What Polkadot wants to achieve through NPoS?

Polkadot gives validator equal voting power in consensus protocol. To reach this, NPoS election mechanism tries to distribute nominators' stake to elected validators as evenly as possible. The objective of this mechanism is to maximize the security of network as most of the validators have equal amount stake, and to achieve two important factors, Fair Representation and Security.

Fair Representation

Polkadot NPoS election system is inspired by Swedish Mathematician Lars Edvard Phragmén's proportional justified representation. That is, when we want n validators to be elected, nominators who stake about (n / total staking) are guaranteed to have at least one of their validators to be elected. Let's look at the illustration about fair representation from Medium.

This illustration is electing 4 validators. First seems unfair representation because it doesn't represent the validator which is backed by 10 Dot nominators. Since 10 Dot is 1/4 of total stake, there should be at least one validator to be elected. However, there is no validator for this nominator, so it is unfair. Rest is fair.

Security

Remember nominators can select up to 16 validators who they trust? Polkadot NPoS mechanism tries to distribute the nominator stake among these validators equal as possible. However, there is a situation where malicious validator is selected with the most backed while other validators with small equal amount. To make this situation difficult, Polkadot reaches Security with the minimum amounts backing of any elected validators. Let's look at the illustration.

Left is level 6 security and right is 9 security. As you can see, second seems more secure since almost all validators have equal amount of backed stake. This is what NPoS mechanism tries to optimize so far.

Conclusion

NPoS is the same way we choose congress man for our country. To make our country a great place, it is our responsibilities to have great interest in whom we pick. Since validators are acting as representatives in Polkadot network, nominators should know who is going to be a good validator and choose carefully. If they choose bad validators, their asset could be slashed, so Nominating is not "set and forget" operation.

There are lots of things happen behind NPoS such as How NPoS handle the nominators and _How Phragmén method is used in NPoS. Later, I am going to handle the details of this mechanism.

Reference

Medium: How Nominated Proof-of-Stake will work in Polkadot
Polkadot-Wiki: Staking

DEV Community: SoYounJeong

Substrate Pallet Code Analysis: Sudo

Overview

Source Code Analysis

Storage

Dispatch Call

Conclusion

Reference

Polkadot Consensus Part 1: BABE Algorithm Analysis

Why do we need consensus?

Polkadot's Consensus

Further Thought

Reference

Polkadot Consensus Part 3: BABE. This is part 3 in our Polkadot… | by Joe Petrowski | Polkadot Network | Medium

Joe Petrowski ・ Dec 19, 2019 ・ Medium

Polkadot's Consensus Protocols · Polkadot Wiki

Polkadot NPoS Overview

Overview

NPoS Scheme

Conclusion

Reference

Joe Petrowski ・ Dec 19, 2019 ・
Medium