DEV Community: Afri

Getting started with Smart Infrastructure in EVE Frontier

Afri — Tue, 08 Oct 2024 16:10:56 +0000

Note: At this stage, there is no publicly available information about the actual EVE Frontier game. The author solely uses all available information on the public documentation and blockchain ledgers for this article. No screenshots or mechanics of the game will be revealed in this post. Assumptions about gameplay are solely made on the base of the sister-game EVE Online.

EVE Frontier is the latest game in development by the Icelandic studio CCP Games. Since not many details are publicly known about this game, let's assume the gameplay is similar to their 21-years old space-MMORPG EVE Online which is an incredible sandbox game, if not one of the most important sandboxes of all times. A sandbox game is basically a world, where you can do whatever you want and there is no straight story arc or given path you have to follow. The entire in-game economy, from mining the smallest unit of rock to building a massive Titan is exclusively driven by players without any NPCs involved.

Now, take this idea of such a massive space sandbox and go a step further: what if you take the sandbox and put it on top of another sandbox that allows players to build the actual game back-end -- creating structures in space, offering missions, establishing security, or whatever you come up with. This is what EVE Frontier is potentially trying to achieve by having an openly programmable blockchain framework underneath. This article explores an example of such a smart assembly and outlines the technical steps required to deploy a functioning component of such in-game infrastructure.

Blockchain Sandbox: Mud & Redstone

EVE Frontier is building on top of Lattice's Redstone layer-2 blockchain, an Ethereum-based roll-up targeted at game developers. A recent playtest was deployed to the Redstone testnet Garnet with chain ID 17069 which rolls up on the Ethereum testnet Holesky. The roll-up features an Ethereum Virtual Machine (EVM) which allows users to program smart contracts with the Solidity programming language.

The blockchain back-end for the game is a set of smart contracts -- referred to as an autonomous world -- that have certain immutable logic and offer the foundation of the game's back-end. However, all the contracts have public APIs and can be extended by your own custom logic. EVE Frontier is using Paradigm's Foundry and Lattice's Mud framework which provides tooling for efficiently developing on-chain content for games.

MUD is provided with not only the game developers but also the users in mind. To retain a great user experience, it employs modern technologies such as meta transactions or account abstraction to make the gameplay as enjoyable as possible.

Now, in EVE Frontier, every player is a cryptographic smart character represented by a pair of SECP256K1 keys, here: an Ethereum account, with a soul-bound token that uniquely identifies your rider. The smart character is basically an ERC-721 non-fungible token (NFT) that is non-transferable. To get an idea how many players tested the latest autonomous world in the alpha playtest, you can check out the token holder list on the blockchain contract 0xdA488adedBEbCC97702F3cc27Ff4A111728721d4 (12,291).

In-game currencies can be created using the ERC-20 token standard. EVE Frontier seemingly introduced a currency EVE that could be required to engage with certain game logic. A list of token holders and transfers for the recent playtest can be found at the token contract 0xb9039385a5c5fFf6C0794829c3149690b310ec8a. 12,326 EVE token holders transferred the token 17,689 times at the time of writing.

But all this is very basic blockchain functionality and not super exiting yet, so let's dive right into world building with smart infrastructure which fully utilizes the tools provided by Mud and potentially makes EVE Frontier a really exciting endeavor.

Smart Infrastructure: Jump Gates

EVE Frontier's smart infrastructure contains a set of smart objects that can be programmed and deployed by players. For the play test, some examples were provided by the Lattice and CCP Games teams: smart turrets that can engage ships in space, smart storage that is a sort of station hanger deployed in space, and smart gates to travel vast distances.

A very powerful asset is a smart gate that allows ships to jump from one star system to another distant system, making travel not only faster but also more secure by evading potential hostile campers that just wait for you to pass through. The following assumes you are familiar with the required tools and workflows as described in EVE Frontier's quick-start documentation. For questions regarding Solidity programming, take a look at the smart-contract documentation.

A smart gate is a system that has one obvious main use: jumping from one gate to another. To control who can jump through, we write a minimal smart contract taken from the templates:

contract MySmartGateSystem is System {
  function canJump(uint256 characterId, uint256 sourceGateId, uint256 destinationGateId) public view returns (bool) {
    return false;
  }
}

Let's walk this through line by line:

MySmartGateSystem is my custom contract that handles the logic of my smart gate. It is extending the class System from the Mud framework which provides a WorldContext. Let's assume for now, that the game developers provide all context so that we can focus on building our own system.
canJump() is a function that handles the permissions using the gate in space. It returns a boolean indicating if a player with characterId can jump from a gate with sourceGateId to destinationGateId.
return false denies access to anyone calling this function. Nobody can jump.

Ok, where do we go from here? We want to use the gate, so we need to start working towards condition handling to allow a player or a set of players to use the gate. One easy way would be to hard-code permissions in the smart contract on chain. Let's take a look.

contract MySmartGateSystem is System {
  function canJump(uint256 characterId, uint256 sourceGateId, uint256 destinationGateId) public view returns (bool) {
    if(characterId == 2112001337)
      return true;
    else
      return false;
  }
}

So, taken straight from the Solidity 101, we implemented a very basic if-else logic. If the character is 2112001337 -- assuming that's me -- then allow me to jump. Everyone else will be denied. Now, this is very easy logic to implement and follow.

However, this is a smart contract that lives on the blockchain and can not be changed after deployment. So, each time we meet someone in game that we like and want to allow them access to our jump-bridge network, we would have to redeploy the contracts and reconfigure the gates, like this:

contract MySmartGateSystem is System {
  function canJump(uint256 characterId, uint256 sourceGateId, uint256 destinationGateId) public view returns (bool) {
    if(characterId == 2112001337)
      return true; // me
    else if(characterId == 2112002342)
      return true; // my friend
    else
      return false;
  }
}

You get the idea. So, what if you want to add 20 friends or maybe give also your corp or alliance access with 100s if not 1000s of players? Managing such a data-set in a smart contract is absolutely not feasible.

Fortunately, the Mud framework provides us with a handy tool called Mud tables. Here, the game provides a table of all characters with some helpful attributes such as belonging to a certain corporation. So, in this case we could allow everyone who is in my corp to also use the gate, like this:

contract MySmartGateSystem is System {
  uint256 public constant MY_CORP_ID = 2000137;
  function canJump(uint256 characterId, uint256 sourceGateId, uint256 destinationGateId) public view returns (bool) {
    uint256 characterCorpId = CharactersTable.getCorpId(characterId);
    if(characterCorpId == MY_CORP_ID)
      return true;
    else
      return false;
  }
}

Let's take a look:

MY_CORP_ID is a numeric identifier of my corp in game. It can be obtained from the character table generated by Mud.
CharactersTable is providing the world context and is generated by the Mud framework filled with data from the actual game.
CharactersTable.getCorpId(characterId) gets the corp identifier from the user trying to access the gate.
We just match characterCorpId to MY_CORP_ID and allow everyone with a matching corp ID to jump the gate.

That's it for the basic logic for now. Next steps would include adding some error handling for invalid inputs such as character Ids or missing corporation records but I'll leave this for the readers to figure out.

Or -- what's your idea such a smart assembly could offer? Rewards for reporting offline infrastructure? Missions to refuel the gates? Let me know!

After getting my hands into building these smart infrastructure components for EVE Frontier, I'm convinced I only scratched the surface of what is possible. Just from reading the documentation I'm extremely hyped about the cooperation of CCP Games and the Lattice team. It proves having a blockchain back-end is much more than just accumulating tokens and trading rare NFTs.

The EVM, Mud, and EVE Frontier give you the power to build your own game as you play it.

Full code:

// SPDX-License-Identifier: MIT
pragma solidity >=0.8.24;

// Mud framework world context imports
import { console } from "forge-std/console.sol";
import { ResourceId } from "@latticexyz/world/src/WorldResourceId.sol";
import { WorldResourceIdLib } from "@latticexyz/world/src/WorldResourceId.sol";
import { IBaseWorld } from "@latticexyz/world/src/codegen/interfaces/IBaseWorld.sol";
import { System } from "@latticexyz/world/src/System.sol";
import { RESOURCE_SYSTEM } from "@latticexyz/world/src/worldResourceTypes.sol";

// Importing the generated character table
import { CharactersTable } from "../../../codegen/tables/CharactersTable.sol";

// My custom smart gate system contract logic
contract MySmartGateSystem is System {

  // A constant holding my corp ID
  uint256 public constant MY_CORP_ID = 2000137;

  // A permission handler for jump requests
  function canJump(uint256 characterId, uint256 sourceGateId, uint256 destinationGateId) public view returns (bool) {

    // Gets the corp id of the player interacting with the assembly
    uint256 characterCorpId = CharactersTable.getCorpId(characterId);

    // Only allow my corp access
    if(characterCorpId == MY_CORP_ID)
      return true;
    else
      return false;
  }
}

Follow the Smart Gate example to learn how to compile and deploy it.

How to merge an Ethereum network right from the genesis block

Afri — Wed, 21 Dec 2022 11:29:47 +0000

The merge is arguably the most significant milestone of the Ethereum research and development roadmap. By attaching a proof-of-stake consensus beacon chain to an already established proof-of-work execution chain, it is possible to swap out the consensus engine on a hot network. However, this is a very tedious task. Wouldn't it be nice to be able to launch an Ethereum network right from the proof-of-stake stage? Here's how!

The following sections outline how to configure an execution-layer client and a consensus-layer client so that they have everything in place to execute the entire merge already in the genesis block. Here, we'll use Geth and Lodestar.

Execution-Layer Configuration

Creating a new genesis for Geth is as simple as running geth dumpgenesis and subsequently modifying fields as needed for the testnet, i.e., for a new devnet or local testnet, we'll run:

geth --sepolia dumpgenesis

This uses the latest Sepolia testnet genesis.

Now, we have to adjust the genesis configuration:

Set a new chainID for replay protection.
Set all protocol upgrades to block 0, including mergeForkBlock and mergeNetsplitBlock.

Set merge total terminal difficulty (TTD) to 0 and the TTD-passed flag to true (terminalTotalDifficulty, terminalTotalDifficultyPassed).

"config":{
  "chainId":39677693,
  "homesteadBlock":0, 
  "eip150Block":0,
  "eip155Block":0,
  "eip158Block":0,
  "byzantiumBlock":0,
  "constantinopleBlock":0,
  "petersburgBlock":0,
  "istanbulBlock":0,
  "muirGlacierBlock":0,
  "berlinBlock":0,
  "londonBlock":0,
  "mergeForkBlock":0,
  "mergeNetsplitBlock":0,
  "terminalTotalDifficulty":0,
  "terminalTotalDifficultyPassed":true
},

Remove the ethash parameters altogether. We don't need any additional pre-merge consensus engine.
Remove the existing alloc and create a custom allocation per our needs.

Adjust the genesis parameters as desired, mainly timestamp, extraData, and nonce, to ensure they do not match an existing genesis. Note that all fields need to be hexadecimal for the Geth genesis file.

  "nonce":"0x44144",
  "timestamp":"0x63A1E047",
  "extraData":"0x887b30d1e617484474521f31a37376760299fa7f15eb4214fce4157744b896c2",
  "gasLimit":"0x1c9c380",
  "difficulty":"0x20000",
  "mixHash":"0x0000000000000000000000000000000000000000000000000000000000000000",
  "coinbase":"0x0000000000000000000000000000000000000000",
  "number":"0x0",
  "gasUsed":"0x0",
  "parentHash":"0x0000000000000000000000000000000000000000000000000000000000000000",

That's it. Now all that is left is the deposit contract. Since we merge from genesis, this has to be part of the genesis state.

For that, we have to take the vanilla deposit contract from the consensus specs: deposit_contract.sol, get the Solidity compiler version 0.6.11, compile the binary of the runtime part and create an empty deposit tree.

Fortunately, the contract's runtime code and initial state remain always the same (see protolambda's notes), so we can copy and paste this into our genesis file.

  "alloc":{
    "0x0420420420420420420420420420420420420420":{
      "balance":"0x0",
      "code": "0x60806040526004361061003f5760003560e01c806301ffc9a71461004457806322895118146100a4578063621fd130146101ba578063c5f2892f14610244575b600080fd5b34801561005057600080fd5b506100906004803603602081101561006757600080fd5b50357fffffffff000000000000000000000000000000000000000000000000000000001661026b565b604080519115158252519081900360200190f35b6101b8600480360360808110156100ba57600080fd5b8101906020810181356401000000008111156100d557600080fd5b8201836020820111156100e757600080fd5b8035906020019184600183028401116401000000008311171561010957600080fd5b91939092909160208101903564010000000081111561012757600080fd5b82018360208201111561013957600080fd5b8035906020019184600183028401116401000000008311171561015b57600080fd5b91939092909160208101903564010000000081111561017957600080fd5b82018360208201111561018b57600080fd5b803590602001918460018302840111640100000000831117156101ad57600080fd5b919350915035610304565b005b3480156101c657600080fd5b506101cf6110b5565b6040805160208082528351818301528351919283929083019185019080838360005b838110156102095781810151838201526020016101f1565b50505050905090810190601f1680156102365780820380516001836020036101000a031916815260200191505b509250505060405180910390f35b34801561025057600080fd5b506102596110c7565b60408051918252519081900360200190f35b60007fffffffff0000000000000000000000000000000000000000000000000000000082167f01ffc9a70000000000000000000000000000000000000000000000000000000014806102fe57507fffffffff0000000000000000000000000000000000000000000000000000000082167f8564090700000000000000000000000000000000000000000000000000000000145b92915050565b6030861461035d576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260268152602001806118056026913960400191505060405180910390fd5b602084146103b6576040517f08c379a000000000000000000000000000000000000000000000000000000000815260040180806020018281038252603681526020018061179c6036913960400191505060405180910390fd5b6060821461040f576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260298152602001806118786029913960400191505060405180910390fd5b670de0b6b3a7640000341015610470576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260268152602001806118526026913960400191505060405180910390fd5b633b9aca003406156104cd576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260338152602001806117d26033913960400191505060405180910390fd5b633b9aca00340467ffffffffffffffff811115610535576040517f08c379a000000000000000000000000000000000000000000000000000000000815260040180806020018281038252602781526020018061182b6027913960400191505060405180910390fd5b6060610540826114ba565b90507f649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c589898989858a8a6105756020546114ba565b6040805160a0808252810189905290819060208201908201606083016080840160c085018e8e80828437600083820152601f017fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe01690910187810386528c815260200190508c8c808284376000838201819052601f9091017fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe01690920188810386528c5181528c51602091820193918e019250908190849084905b83811015610648578181015183820152602001610630565b50505050905090810190601f1680156106755780820380516001836020036101000a031916815260200191505b5086810383528881526020018989808284376000838201819052601f9091017fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0169092018881038452895181528951602091820193918b019250908190849084905b838110156106ef5781810151838201526020016106d7565b50505050905090810190601f16801561071c5780820380516001836020036101000a031916815260200191505b509d505050505050505050505050505060405180910390a1600060028a8a600060801b604051602001808484808284377fffffffffffffffffffffffffffffffff0000000000000000000000000000000090941691909301908152604080517ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0818403018152601090920190819052815191955093508392506020850191508083835b602083106107fc57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe090920191602091820191016107bf565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610859573d6000803e3d6000fd5b5050506040513d602081101561086e57600080fd5b5051905060006002806108846040848a8c6116fe565b6040516020018083838082843780830192505050925050506040516020818303038152906040526040518082805190602001908083835b602083106108f857805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe090920191602091820191016108bb565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610955573d6000803e3d6000fd5b5050506040513d602081101561096a57600080fd5b5051600261097b896040818d6116fe565b60405160009060200180848480828437919091019283525050604080518083038152602092830191829052805190945090925082918401908083835b602083106109f457805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe090920191602091820191016109b7565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610a51573d6000803e3d6000fd5b5050506040513d6020811015610a6657600080fd5b5051604080516020818101949094528082019290925280518083038201815260609092019081905281519192909182918401908083835b60208310610ada57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101610a9d565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610b37573d6000803e3d6000fd5b5050506040513d6020811015610b4c57600080fd5b50516040805160208101858152929350600092600292839287928f928f92018383808284378083019250505093505050506040516020818303038152906040526040518082805190602001908083835b60208310610bd957805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101610b9c565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610c36573d6000803e3d6000fd5b5050506040513d6020811015610c4b57600080fd5b50516040518651600291889160009188916020918201918291908601908083835b60208310610ca957805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101610c6c565b6001836020036101000a0380198251168184511680821785525050505050509050018367ffffffffffffffff191667ffffffffffffffff1916815260180182815260200193505050506040516020818303038152906040526040518082805190602001908083835b60208310610d4e57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101610d11565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610dab573d6000803e3d6000fd5b5050506040513d6020811015610dc057600080fd5b5051604080516020818101949094528082019290925280518083038201815260609092019081905281519192909182918401908083835b60208310610e3457805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101610df7565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015610e91573d6000803e3d6000fd5b5050506040513d6020811015610ea657600080fd5b50519050858114610f02576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260548152602001806117486054913960600191505060405180910390fd5b60205463ffffffff11610f60576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260218152602001806117276021913960400191505060405180910390fd5b602080546001019081905560005b60208110156110a9578160011660011415610fa0578260008260208110610f9157fe5b0155506110ac95505050505050565b600260008260208110610faf57fe5b01548460405160200180838152602001828152602001925050506040516020818303038152906040526040518082805190602001908083835b6020831061102557805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101610fe8565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa158015611082573d6000803e3d6000fd5b5050506040513d602081101561109757600080fd5b50519250600282049150600101610f6e565b50fe5b50505050505050565b60606110c26020546114ba565b905090565b6020546000908190815b60208110156112f05781600116600114156111e6576002600082602081106110f557fe5b01548460405160200180838152602001828152602001925050506040516020818303038152906040526040518082805190602001908083835b6020831061116b57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0909201916020918201910161112e565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa1580156111c8573d6000803e3d6000fd5b5050506040513d60208110156111dd57600080fd5b505192506112e2565b600283602183602081106111f657fe5b015460405160200180838152602001828152602001925050506040516020818303038152906040526040518082805190602001908083835b6020831061126b57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0909201916020918201910161122e565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa1580156112c8573d6000803e3d6000fd5b5050506040513d60208110156112dd57600080fd5b505192505b6002820491506001016110d1565b506002826112ff6020546114ba565b600060401b6040516020018084815260200183805190602001908083835b6020831061135a57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0909201916020918201910161131d565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790527fffffffffffffffffffffffffffffffffffffffffffffffff000000000000000095909516920191825250604080518083037ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff8018152601890920190819052815191955093508392850191508083835b6020831061143f57805182527fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe09092019160209182019101611402565b51815160209384036101000a7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01801990921691161790526040519190930194509192505080830381855afa15801561149c573d6000803e3d6000fd5b5050506040513d60208110156114b157600080fd5b50519250505090565b60408051600880825281830190925260609160208201818036833701905050905060c082901b8060071a60f81b826000815181106114f457fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060061a60f81b8260018151811061153757fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060051a60f81b8260028151811061157a57fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060041a60f81b826003815181106115bd57fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060031a60f81b8260048151811061160057fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060021a60f81b8260058151811061164357fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060011a60f81b8260068151811061168657fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a9053508060001a60f81b826007815181106116c957fe5b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916908160001a90535050919050565b6000808585111561170d578182fd5b83861115611719578182fd5b505082019391909203915056fe4465706f736974436f6e74726163743a206d65726b6c6520747265652066756c6c4465706f736974436f6e74726163743a207265636f6e7374727563746564204465706f7369744461746120646f6573206e6f74206d6174636820737570706c696564206465706f7369745f646174615f726f6f744465706f736974436f6e74726163743a20696e76616c6964207769746864726177616c5f63726564656e7469616c73206c656e6774684465706f736974436f6e74726163743a206465706f7369742076616c7565206e6f74206d756c7469706c65206f6620677765694465706f736974436f6e74726163743a20696e76616c6964207075626b6579206c656e6774684465706f736974436f6e74726163743a206465706f7369742076616c756520746f6f20686967684465706f736974436f6e74726163743a206465706f7369742076616c756520746f6f206c6f774465706f736974436f6e74726163743a20696e76616c6964207369676e6174757265206c656e677468a26469706673582212201dd26f37a621703009abf16e77e69c93dc50c79db7f6cc37543e3e0e3decdc9764736f6c634300060b0033",
      "storage": {
        "0x0000000000000000000000000000000000000000000000000000000000000022": "0xf5a5fd42d16a20302798ef6ed309979b43003d2320d9f0e8ea9831a92759fb4b",
        "0x0000000000000000000000000000000000000000000000000000000000000023": "0xdb56114e00fdd4c1f85c892bf35ac9a89289aaecb1ebd0a96cde606a748b5d71",
        "0x0000000000000000000000000000000000000000000000000000000000000024": "0xc78009fdf07fc56a11f122370658a353aaa542ed63e44c4bc15ff4cd105ab33c",
        "0x0000000000000000000000000000000000000000000000000000000000000025": "0x536d98837f2dd165a55d5eeae91485954472d56f246df256bf3cae19352a123c",
        "0x0000000000000000000000000000000000000000000000000000000000000026": "0x9efde052aa15429fae05bad4d0b1d7c64da64d03d7a1854a588c2cb8430c0d30",
        "0x0000000000000000000000000000000000000000000000000000000000000027": "0xd88ddfeed400a8755596b21942c1497e114c302e6118290f91e6772976041fa1",
        "0x0000000000000000000000000000000000000000000000000000000000000028": "0x87eb0ddba57e35f6d286673802a4af5975e22506c7cf4c64bb6be5ee11527f2c",
        "0x0000000000000000000000000000000000000000000000000000000000000029": "0x26846476fd5fc54a5d43385167c95144f2643f533cc85bb9d16b782f8d7db193",
        "0x000000000000000000000000000000000000000000000000000000000000002a": "0x506d86582d252405b840018792cad2bf1259f1ef5aa5f887e13cb2f0094f51e1",
        "0x000000000000000000000000000000000000000000000000000000000000002b": "0xffff0ad7e659772f9534c195c815efc4014ef1e1daed4404c06385d11192e92b",
        "0x000000000000000000000000000000000000000000000000000000000000002c": "0x6cf04127db05441cd833107a52be852868890e4317e6a02ab47683aa75964220",
        "0x000000000000000000000000000000000000000000000000000000000000002d": "0xb7d05f875f140027ef5118a2247bbb84ce8f2f0f1123623085daf7960c329f5f",
        "0x000000000000000000000000000000000000000000000000000000000000002e": "0xdf6af5f5bbdb6be9ef8aa618e4bf8073960867171e29676f8b284dea6a08a85e",
        "0x000000000000000000000000000000000000000000000000000000000000002f": "0xb58d900f5e182e3c50ef74969ea16c7726c549757cc23523c369587da7293784",
        "0x0000000000000000000000000000000000000000000000000000000000000030": "0xd49a7502ffcfb0340b1d7885688500ca308161a7f96b62df9d083b71fcc8f2bb",
        "0x0000000000000000000000000000000000000000000000000000000000000031": "0x8fe6b1689256c0d385f42f5bbe2027a22c1996e110ba97c171d3e5948de92beb",
        "0x0000000000000000000000000000000000000000000000000000000000000032": "0x8d0d63c39ebade8509e0ae3c9c3876fb5fa112be18f905ecacfecb92057603ab",
        "0x0000000000000000000000000000000000000000000000000000000000000033": "0x95eec8b2e541cad4e91de38385f2e046619f54496c2382cb6cacd5b98c26f5a4",
        "0x0000000000000000000000000000000000000000000000000000000000000034": "0xf893e908917775b62bff23294dbbe3a1cd8e6cc1c35b4801887b646a6f81f17f",
        "0x0000000000000000000000000000000000000000000000000000000000000035": "0xcddba7b592e3133393c16194fac7431abf2f5485ed711db282183c819e08ebaa",
        "0x0000000000000000000000000000000000000000000000000000000000000036": "0x8a8d7fe3af8caa085a7639a832001457dfb9128a8061142ad0335629ff23ff9c",
        "0x0000000000000000000000000000000000000000000000000000000000000037": "0xfeb3c337d7a51a6fbf00b9e34c52e1c9195c969bd4e7a0bfd51d5c5bed9c1167",
        "0x0000000000000000000000000000000000000000000000000000000000000038": "0xe71f0aa83cc32edfbefa9f4d3e0174ca85182eec9f3a09f6a6c0df6377a510d7",
        "0x0000000000000000000000000000000000000000000000000000000000000039": "0x31206fa80a50bb6abe29085058f16212212a60eec8f049fecb92d8c8e0a84bc0",
        "0x000000000000000000000000000000000000000000000000000000000000003a": "0x21352bfecbeddde993839f614c3dac0a3ee37543f9b412b16199dc158e23b544",
        "0x000000000000000000000000000000000000000000000000000000000000003b": "0x619e312724bb6d7c3153ed9de791d764a366b389af13c58bf8a8d90481a46765",
        "0x000000000000000000000000000000000000000000000000000000000000003c": "0x7cdd2986268250628d0c10e385c58c6191e6fbe05191bcc04f133f2cea72c1c4",
        "0x000000000000000000000000000000000000000000000000000000000000003d": "0x848930bd7ba8cac54661072113fb278869e07bb8587f91392933374d017bcbe1",
        "0x000000000000000000000000000000000000000000000000000000000000003e": "0x8869ff2c22b28cc10510d9853292803328be4fb0e80495e8bb8d271f5b889636",
        "0x000000000000000000000000000000000000000000000000000000000000003f": "0xb5fe28e79f1b850f8658246ce9b6a1e7b49fc06db7143e8fe0b4f2b0c5523a5c",
        "0x0000000000000000000000000000000000000000000000000000000000000040": "0x985e929f70af28d0bdd1a90a808f977f597c7c778c489e98d3bd8910d31ac0f7"
      }
    }
  }

So, we add the binary and storage tuples to the allocation of the genesis file.

Once set, we initialize as many execution clients as desired, using geth init. Let's start with two:

geth init \
  --datadir "./data/execution/0" \
  ./execution/genesis.json

geth init \
  --datadir "./data/execution/1" \
  ./execution/genesis.json

Lastly, we get the genesis hash to be used as a reference for the consensus client setup later. For that, we run an execution node and get the genesis from it, e.g., attach to it and query eth.getBlockByNumber(0).

geth \
  --networkid 39677693 \
  --datadir "./data/execution/0" \
  console

Note down the genesis hash for the consensus-layer configuration later and also get the node's enode address to connect other execution-layer clients. Let's connect another one right away.

geth \
  --networkid 39677693 \
  --datadir "./data/execution/1" \
  --authrpc.port 8651 \
  --port 31303 \
  --bootnodes "enode://39dc4161c55076d1245c7c8383996606e1d1d4a9006e58454bd8e2c139970269fa682560bfed2087bf1d1f5d5fc0d40f143b46398a8eaee343b43136f373f091@127.0.0.1:30303"

Persist the bootnodes in a text file for later use, e.g., bootnodes.txt.

That's our execution-layer configuration. It currently runs two nodes locally that do nothing except wait for a consensus client to provide blocks.

Consensus-Layer Configuration

To create a new consensus-layer configuration, get the latest mainnet.yaml config from the consensus specs.

Come up with a unique CONFIG_NAME, i.e., the name of our testnet, here: mergednet.
Set the TTD to 0 and the terminal hash to the genesis hash we previously extracted from the execution-layer client.
Set the minimum genesis time to the genesis time of the execution layer or any desired time greater than that. Also, create a sensible genesis delay, e.g., two hours. We don't want to wait seven days!
Set the fork versions to something custom to not conflict with mainnet, e.g., 0x00677693 for genesis, 0x01677693 for Altair, etc. Also, consider setting all upgrades to activate on genesis epoch 0.
Set the deposit chain and network ID to the previously created execution-layer network and match the deposit contract address to the one in the genesis, here: 39677693, 0x0420420420420420420420420420420420420420.

In addition to the config.yaml for the consensus layer, we also create auxiliary files for the deposit contract:

deploy_block.txt containing the block number 0
deposit_contract_block.txt containing the execution genesis hash
deposit_contract.txt containing the deposit-contract address from the genesis allocation, here: 0x0420420420420420420420420420420420420420

Lastly, before generating the genesis, we need to create validators that kick off the chain right from the start.

We'll generate BIP-39 mnemonics for everyone who is going to run genesis validators and store them in a mnemonics.yaml file. Take a look at protolambda's notes for mnemonic-yaml formatting: protolambda/eth2-testnet-genesis#mnemonics.
Update the MIN_GENESIS_ACTIVE_VALIDATOR_COUNT to something sensible; we don't need 16k validators for our network yet. If you plan to run eight validators, set it to 8.
Lastly, use the eth2 testnet genesis tool to generate the genesis.ssz state for a merge configuration:

eth2-testnet-genesis merge \
  --config "./consensus/config.yaml" \
  --eth1-config "./execution/genesis.json" \
  --mnemonics "./consensus/mnemonic.yaml" \
  --state-output "./consensus/genesis.ssz" \
  --tranches-dir "./consensus/tranches"

The genesis state will contain the initial genesis validators, and there is no need to conduct actual deposits. To add more validators later, use the deposit contract at any time.

That's all for the configuration. We can run beacon nodes now and allow them to network and discover peers before genesis. Note that the network needs at least two peered beacon nodes; otherwise, the nodes will act as if they are offline and refuse to author blocks!

lodestar beacon \
  --suggestedFeeRecipient "0xCaA29806044A08E533963b2e573C1230A2cd9a2d" \
  --execution.urls "http://127.0.0.1:8551" \
  --jwt-secret "./data/execution/0/geth/jwtsecret" \
  --dataDir "./data/consensus/0" \
  --paramsFile "./consensus/config.yaml" \
  --genesisStateFile "./consensus/genesis.ssz" \
  --enr.ip 127.0.0.1

Note we are setting the ENR IP to whatever we need (here: localhost) so that we can use the ENR as the initial bootnode record. Get it from the identity endpoint:

curl http://localhost:9596/eth/v1/node/identity

All ENRs can be persisted in a file such as bootnodes.txt that we can pass to the beacon-chain clients later to bootstrap the networking.

lodestar beacon \
  --suggestedFeeRecipient "0xCaA29806044A08E533963b2e573C1230A2cd9a2d" \
  --execution.urls "http://127.0.0.1:8651" \
  --jwt-secret "./data/execution/1/geth/jwtsecret" \
  --dataDir "./data/consensus/1" \
  --paramsFile "./consensus/config.yaml" \
  --genesisStateFile "./consensus/genesis.ssz" \
  --enr.ip 127.0.0.1 \
  --rest.port 9696 \
  --port 9100 \
  --network.connectToDiscv5Bootnodes true \
  --bootnodes "enr:-Ku4QFqWLadqfhKv7669IBdUwAilxPt9khrHqXVTqZjYdYtyNGTs4qeZslPf5Jx61QrKnMZ6-AcSgvu4zEcmSnnUrIIFh2F0dG5ldHOIAAAAAAAAAACEZXRoMpDYGrlYAmd2k___________gmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQIK5OOTyDgyoNl6TIqkWyap7VTqnIzlq40kE4N-250mV4N0Y3CCIyg"

We do this for as many beacon nodes as we want. Note that we need to enable discv5 bootnodes for our local network; otherwise, our clients will not peer.

That's it for our consensus-layer configuration. We have two Geth execution nodes running with two Lodestar beacon nodes connected with each other. Now, we have to launch the network.

Launch the Network

We have both execution-layer and consensus-layer clients running already. We can easily make sure they are running correctly by reading the logs. If they fail to authenticate or connect, there will be errors.

However, to kick off the network, we must ensure the validators are loaded and connected before genesis. If we fail to generate a chain within the first 32 slots, the clients will remain forever in a syncing state, potentially preventing long-range attacks. Therefore, if your beacon node is already past the 32nd slot, bump the genesis delay and redo the consensus-layer configuration.

We can use the deposit CLI to generate validator keypairs. Run ./deposit existing-mnemonic and follow the instructions.

This generates deposit data and the actual keypairs. The deposit data can be discarded as we don't need to conduct any deposits. The keys can be imported to our validator client. Make sure also to provide a password.

lodestar validator \
  --dataDir "./data/consensus/0" \
  --suggestedFeeRecipient "0xCaA29806044A08E533963b2e573C1230A2cd9a2d" \
  --graffiti "YOLO MERGEDNET GETH LODESTAR" \
  --paramsFile "./consensus/config.yaml" \
  --importKeystores "./consensus/validator_keys" \
  --importKeystoresPassword "./consensus/validator_keys/password.txt"

Once the keys and passwords are imported, the validator client will wait to take on its duties. Once the minimum genesis time and genesis delay is passed, it will propose blocks and attestations.

That's it. That's our new Ethereum network, merged right from genesis.

Additional Resources

Credit goes to Gajinder, Parithosh, and Protolamda for the valuable pointers, tools, and documentation. Thank you so much for your contributions!

github/protolambda/merge-genesis-tools: Testnet tooling to create a merge state at genesis.
github/ethereum/consensus-specs: Ethereum proof-of-stake consensus specifications, including mainnet templates and deposit contract.
notes/protolambda/merge-devnet-setup-guide: Multi-client post-merge Ethereum devnet setup by protolambda.
github/ethereumjs/consensus-deployment-ansible: Ansible playbooks by Parithosh that allow us to do everything we did in this tutorial with one click ;)
github/protolambda/eth2-testnet-genesis: Tool to create a genesis state for an beacon-chain testnet.
github/ethereum/staking-deposit-cli: Secure key generation for beacon-chain deposits.
github/ethereum/go-ethereum: Official Go implementation of the Ethereum protocol (execution).
github/ChainSafe/lodestar: TypeScript implementation of an Ethereum consensus layer.
github/eth-clients/holesovice: Preview: the first public merged-from-genesis Ethereum testnet.

Now that you scrolled to the end of this article, I will reward you with the entire configuration used in this tutorial: github.com/q9f/mergednet - I pedantically created one commit for each step.

Thanks for reading!

Finally authenticating Rails users with MetaMask

Afri — Tue, 01 Mar 2022 13:54:22 +0000

It's not a secret that passwords are a relic from a different century. However, modern cryptography provides us with far better means to authenticate with applications, such as Ethereum's Secp256k1 public-private key pairs. This article is a complete step-by-step deep-dive to securely establish a Ruby-on-Rails user session with an Ethereum account instead of a password. In addition, it aims to explain how it's done by providing code samples and expands on the security implications. (For the impatient, the entire code is available on Github at ethereum-on-rails.)

Web3 Concepts

This post comes with some technical depth and introduces mechanics that are relatively new concepts and require you to understand some context. However, if you already know what Web3 is, scroll down to the next section.

Web3 is a relatively new term that introduces us to a new generation of web applications after Web 1.0 and 2.0. It's beyond the scope of this article to explain the concepts of Web3. However, it's essential to understand that web components and services are no longer hosted on servers. Instead, web applications embed content from decentralized storage solutions, such as IPFS, or consensus protocols, such as Ethereum.

Notably, there are different ways to integrate such components in web applications. However, since the most prominent way to access the web is a web browser, most Web3 content can be easily accessed through browser extensions. For example, data hosted on IPFS can be retrieved through local or remote nodes using an extension called IPFS Companion. In addition, for blockchains such as Ethereum, there are extensions like MetaMask.

The benefit of such an Ethereum extension is the different ways of accessing blockchain states and the ability for users to manage their Ethereum accounts. And this is what we will utilize for this tutorial: an Ethereum account in a MetaMask browser extension connecting to your Ruby-on-Rails web application to authenticate a user session securely.

Authentication Process Overview

Before diving in and creating a new Rails app, let's take a look at the components we'll need throughout the tutorial.

We need to create a user model that includes fields for the Ethereum address of the user and a random nonce that the user will sign later during authentication for security reasons.
We'll create an API endpoint that allows fetching the random nonce for a user's Ethereum address from the backend to be available for signing in the frontend.
In the browser, we'll generate a custom message containing the website title, the user's nonce, and a current timestamp that the user has to sign with their browser extension using their Ethereum account.
All these bits, the signature, the message, and the user's account are cryptographically verified in the Rails backend.
If this succeeds, we'll create a new authenticated user session and rotate the user's nonce to prevent signature spoofing for future logins.

Let's get started.

Rails' User Model

We'll use a fresh Rails 7 installation without additional modules or custom functionality. Just install Rails and get a new instance according to the docs.

rails new myapp
cd myapp

Create an app/models/user.rb first, which will define the bare minimum required for our user model.

class User < ApplicationRecord
  validates :eth_address, presence: true, uniqueness: true
  validates :eth_nonce, presence: true, uniqueness: true
  validates :username, presence: true, uniqueness: true
end

Note that we no longer care about passwords, email addresses, or other fields. Of course, you may add any arbitrary field you like, but these three fields are essential for an Ethereum authentication:

The username is a human-friendly string allowing users to identify themself with a nym.
The user's Ethereum account address is to authenticate with your application.
The nonce is a random secret in the user database schema used to prevent signature spoofing (more on that later).

User Controller `#create`

The controllers are powerful Rails tools to handle your routes and application logic. Here, we will implement creating new user accounts with an Ethereum address in app/controllers/users_controller.rb.

require "eth"

def create
  # only proceed with pretty names
  if @user and @user.username and @user.username.size > 0
    # create random nonce
    @user.eth_nonce = SecureRandom.uuid
    # only proceed with eth address
    if @user.eth_address
      # make sure the eth address is valid
      if Eth::Address.new(@user.eth_address).valid?
        # save to database
        if @user.save
          # if user is created, congratulations, send them to login
          redirect_to login_path, notice: "Successfully created an account, you may now log in."
        end
      end
    end
  end
end

The Users controller is solely used for creating new users.

It generates an initial random nonce with SecureRandom.uuid.
It ensures the user picks a name.
It takes the eth_address from the sign-up view (more on that later).
It guarantees the eth_address is a valid Ethereum address.
It creates a new user and saves it to the database with the given attributes.

We are using the eth gem to validate the address field.

Be aware that we do not require any signature to reduce complexity and increase the accessibility of this tutorial. It is, however, strongly recommended to unify the login and sign-up process to prevent unnecessary spam in the user database, i.e., if a user with the given address does not exist, create it.

Connecting to MetaMask

We already taught our Rails backend what a User object looks like (model) and how to handle logic (controller). However, two components are missing to make this work: a new-user view rendering the sign-up form and some JavaScript to manage the frontend logic.

For the sign-up form, add a form_for @user to the app/views/users/new.html.erb view.

<%= form_for @user, url: signup_path do |form| %>
  <%= form.label "Name" %>
  <%= form.text_field :username %> <br />
  <%= form.text_field :eth_address, readonly: true, class: "eth_address" %> <br />
<% end %>
<button class="eth_connect">Sign-up with Ethereum</button>
<%= javascript_pack_tag "users_new" %>

We'll allow the user to fill in the :username field but make the :eth_address field read-only because this will be filled in by the browser extension. We could even add some CSS to hide it.

Lastly, the eth_connect button triggers the JavaScript to connect to MetaMask and query the user's Ethereum account. But, first, let's take a look at app/javascript/packs/users_new.js.

// the button to connect to an ethereum wallet
const buttonEthConnect = document.querySelector('button.eth_connect');
// the read-only eth address field, we process that automatically
const formInputEthAddress = document.querySelector('input.eth_address');
// get the user form for submission later
const formNewUser = document.querySelector('form.new_user');
// only proceed with ethereum context available
if (typeof window.ethereum !== 'undefined') {
  buttonEthConnect.addEventListener('click', async () => {
    // request accounts from ethereum provider
    const accounts = await ethereum.request({ method: 'eth_requestAccounts' });
    // populate and submit form
    formInputEthAddress.value = accounts[0];
    formNewUser.submit();
  });
}

The JavaScript contains the following logic:

It ensures an Ethereum context is available.
It adds a click-event listener to the connect button.
It requests accounts from the available Ethereum wallet: method: 'eth_requestAccounts'
It adds the eth_address to the form and submits it.

Now, we have a Rails application with the basic User logic implemented. But how do we authenticate the users finally?

User Sessions

The previous sections were an introduction, preparing a Rails application to handle users with the schema we need. Now, we are getting to the core of the authentication: Users are the prerequisite; logging in a user requires a Session. Let's take a look at the app/controllers/sessions_controller.rb.

require "eth"
require "time"

def create
  # users are indexed by eth address here
  user = User.find_by(eth_address: params[:eth_address])
  # if the user with the eth address is on record, proceed
  if user.present?
    # if the user signed the message, proceed
    if params[:eth_signature]
      # the message is random and has to be signed in the ethereum wallet
      message = params[:eth_message]
      signature = params[:eth_signature]
      # note, we use the user address and nonce from our database, not from the form
      user_address = user.eth_address
      user_nonce = user.eth_nonce
      # we embedded the time of the request in the signed message and make sure
      # it's not older than 5 minutes. expired signatures will be rejected.
      custom_title, request_time, signed_nonce = message.split(",")
      request_time = Time.at(request_time.to_f / 1000.0)
      expiry_time = request_time + 300
      # also make sure the parsed request_time is sane
      # (not nil, not 0, not off by orders of magnitude)
      sane_checkpoint = Time.parse "2022-01-01 00:00:00 UTC"
      if request_time and request_time > sane_checkpoint and Time.now < expiry_time
        # enforce that the signed nonce is the one we have on record
        if signed_nonce.eql? user_nonce
          # recover address from signature
          signature_pubkey = Eth::Signature.personal_recover message, signature
          signature_address = Eth::Util.public_key_to_address signature_pubkey
          # if the recovered address matches the user address on record, proceed
          # (uses downcase to ignore checksum mismatch)
          if user_address.downcase.eql? signature_address.to_s.downcase
            # if this is true, the user is cryptographically authenticated!
            session[:user_id] = user.id
            # rotate the random nonce to prevent signature spoofing
            user.eth_nonce = SecureRandom.uuid
            user.save
            # send the logged in user back home
            redirect_to root_path, notice: "Logged in successfully!"
          end
        end
      end
    end
  end
end

The controller does the following.

It finds the user by eth_address provided by the Ethereum wallet.
It ensures the user exists in the database by looking up the address.
It guarantees the user signed an eth_message to authenticate (more on that later).
It ensures the eth_signature field is not expired (older than five minutes).
It assures the signed eth_nonce matches the one in our database.
It recovers the public key and address from the signature.
It ensures the recovered address matches the address in the database.
It logs the user in if all the above is true.
If all of the above is true, it rotates a new nonce for future logins.

The code above, the #create-session controller, contains all security checks for the backend authentication. To successfully log in, all assessments need to pass.

Now that we have the controller, we still need a view and the frontend JavaScript logic. The view needs the form and the button in app/views/sessions/new.html.erb.

<%= form_tag "/login", class: "new_session" do %>
  <%= text_field_tag :eth_message, "", readonly: true, class: "eth_message" %> <br />
  <%= text_field_tag :eth_address, "", readonly: true, class: "eth_address" %> <br />
  <%= text_field_tag :eth_signature, "", readonly: true, class: "eth_signature" %> <br />
<% end %>
<button class="eth_connect">Login with Ethereum</button>
<%= javascript_pack_tag "sessions_new" %>

The login form only contains three read-only fields: address, message, and signature. We can hide them and let JavaScript handle the content. The user will only interact with the button and the browser extension. So, last but not least, we'll take a look at our frontend logic in app/javascript/packs/sessions_new.js.

// the button to connect to an ethereum wallet
const buttonEthConnect = document.querySelector('button.eth_connect');
// the read-only eth fields, we process them automatically
const formInputEthMessage = document.querySelector('input.eth_message');
const formInputEthAddress = document.querySelector('input.eth_address');
const formInputEthSignature = document.querySelector('input.eth_signature');
// get the new session form for submission later
const formNewSession = document.querySelector('form.new_session');
// only proceed with ethereum context available
if (typeof window.ethereum !== 'undefined') {
  buttonEthConnect.addEventListener('click', async () => {
    // request accounts from ethereum provider
    const accounts = await requestAccounts();
    const etherbase = accounts[0];
    // sign a message with current time and nonce from database
    const nonce = await getUuidByAccount(etherbase);
    if (nonce) {
      const customTitle = "Ethereum on Rails";
      const requestTime = new Date().getTime();
      const message = customTitle + "," + requestTime + "," + nonce;
      const signature = await personalSign(etherbase, message);
      // populate and submit form
      formInputEthMessage.value = message;
      formInputEthAddress.value = etherbase;
      formInputEthSignature.value = signature;
      formNewSession.submit();
    }
  });
}

That's a lot to digest, so let's look at what the script does, step by step.

It, again, ensures an Ethereum context is available.
It adds a click-event listener to the eth_connect button.
It requests accounts from the available Ethereum wallet: method: 'eth_requestAccounts'
It requests the nonce belonging to the account from the API/v1 (more on that later).
It generates a message containing the site's title, the request time, and the nonce from the API/v1.
It requests the user to sign the message: method: 'personal_sign', params: [ message, account ]
It populates the form with address, message, and signature and submits it.

Putting aside the API/v1 (for now), we have everything in place: The Rails application crafts a custom message containing a random nonce and a timestamp. Then, the frontend requests the user to sign the payload with their Ethereum account. The following snippet shows the relevant JavaScript for requesting accounts and signing the message.

// request ethereum wallet access and approved accounts[]
async function requestAccounts() {
  const accounts = await ethereum.request({ method: 'eth_requestAccounts' });
  return accounts;
}

// request ethereum signature for message from account
async function personalSign(account, message) {
  const signature = await ethereum.request({ method: 'personal_sign', params: [ message, account ] });
  return signature;
}

Once the message is signed, both the message and the signature, along with the Ethereum account's address, get passed to the Rails backend for verification. If all backend checks succeed (see session controller above), we consider the user authenticated.

Back and forth

Let's quickly recap. We have a user model containing address, nonce, and name for every user of our Rails application. To create a user, we allow the user to pick a nym, ask the browser extension for the user's Ethereum address and roll a random nonce (here: UUID) for the user database. To authenticate, we let the user sign a message containing a custom string (here: site title), the user's nonce, and a timestamp to force the signature to expire. If the signature matches the Ethereum account and nonce on the record and is not expired, we consider the user cryptographically authenticated.

But one thing is missing. So far, both creating a user and authenticating a new session was a one-way operation, passing data from the frontend to the backend for validation. However, to sign the required nonce from the user database, we need a way for the frontend to access the user's nonce. For that, we create a public API endpoint that allows querying the eth_nonce from the user database by the eth_address key. Let's take a look at app/controllers/api/v1/users_controller.rb.

require "eth"

class Api::V1::UsersController < ApiController
  # creates a public API that allows fetching the user nonce by address
  def show
    user = nil
    response = nil
    # checks the parameter is a valid eth address
    params_address = Eth::Address.new params[:id]
    if params_address.valid?
      # finds user by valid eth address (downcase to prevent checksum mismatchs)
      user = User.find_by(eth_address: params[:id].downcase)
    end
    # do not expose full user object; just the nonce
    if user and user.id > 0
      response = [eth_nonce: user.eth_nonce]
    end
    # return response if found or nil in case of mismatch
    render json: response
  end
end

The #show controller gets a user by eth_address from the database and returns the eth_nonce or nil if it does not exist.

GET /api/v1/users/${eth_account}
It ensures the eth_account parameter is a valid Ethereum address to filter out random requests.
It finds a user in the database by eth_account key.
It returns only the eth_nonce as JSON.
It returns nothing if it fails any of the above steps.

The frontend can use some JavaScript to fetch this during authentication.

// get nonce from /api/v1/users/ by account
async function getUuidByAccount(account) {
  const response = await fetch("/api/v1/users/" + account);
  const nonceJson = await response.json();
  if (!nonceJson) return null;
  const uuid = nonceJson[0].eth_nonce;
  return uuid;
}

And that's it. So now we have all pieces in place. Run your Rails application and test it out!

bundle install
bin/rails db:migrate
bin/rails server

What did I just read?

To recap, an Ethereum account is a public-private key pair (very similar to SSH, OTR, or PGP keys) that can be used to authenticate a user on any web application without any need for an email, a password, or other gimmicks.

Our application identifies the user not by its name but by the public Ethereum address belonging to their account. By cryptographically signing a custom message containing a user secret and a timestamp, the user can prove that they control the Ethereum account belonging to the user on the record.

A valid, not expired signature matching the nonce and address of the user allows us to grant the user access to our Rails application securely.

Security Considerations

One might wonder, is this secure?

Generally speaking, having an Ethereum account in a browser extension is comparable with a password manager in a browser extension from an operational security standpoint. The password manager fills the login form with your email and password, whereas the Ethereum wallet shares your address and the signature you carefully approved.

From a technical perspective, it's slightly more secure as passwords can be easier compromised than signatures. For example, a website that tricks you into believing they are your bank can very well steal your bank account credentials. This deception is called phishing, and once your email and password are compromised, malicious parties can attempt to log in to all websites where they suspect you of having the same credentials.

Phishing Ethereum signatures is also possible, but due to the very limited validity of the signature both in time and scope, it's more involved. The user nonce in the backend gets rotated with each login attempt, making a signature valid only once. By adding a timestamp to the signed message, applications can also reduce attackers' window of opportunity to just a few minutes.

Isn't there a standard for that?

There is: EIP-4361 tries to standardize the message signed by the user. Check out the Sign-in with Ethereum (SIWE) project.

This article is considered educational material and does not use the SIWE-libraries to elaborate on more detailed steps and components. However, it's recommended to check out the Rails SIWE examples for production.

Does this make sense? Please let me know in the comments! Thanks for reading!

Further resources

q9f/ethereum-on-rails: All the code from this tutorial in one place.
EIP-4361: Sign-In with Ethereum: an attempt to standardize the message you sign for authentication.
Sign-In with Ethereum - Ruby Library and Rails Examples Release: an implementation of EIP-4361 in Ruby-on-Rails.
One-click Login with Blockchain - A MetaMask Tutorial by my former colleague Amaury Martiny.

Finally Understanding Ethereum Accounts

Afri — Tue, 15 Feb 2022 15:39:55 +0000

Ethereum is a public blockchain network that can be accessed with various types of accounts. Similar to Bitcoin, the underlying cryptography uses the SECP256K1 elliptic curve. But what does this mean? What is an account? What is a key? What is an address, and why is it checksummed? A crash-curse in Ruby and Crystal (using the eth gem and the secp256k1 shard).

Private-Public Keypairs

An Ethereum account is a SECP256K1 keypair. "SECP256K1" is just the name of the specific elliptic curve we are using. The name or specification of that curve is not essential to understanding how keypair-cryptography works, and there are many more curves with many different names and parameters.

A keypair contains a private and a public part. The private portion, also secret, or private key gives you access to the account. A private key is simply a number: 1 is a private key, 137 is a private key, 29863245 is another. You should not use any of them for security reasons as they are easily guessed, but as you can see, there is no magic behind a key at all.

# A private key is really just a random number.
# To generate a random private key that nobody 
# can guess in Ruby, get 32 random bytes.
require "securerandom"
secret = SecureRandom.hex 32
# => "bec52dffb33ec1f4d629f88232796e898f4294079c5894a6645e8a4f7261fabe"

Understanding the public part, or public key, is more involved. It is a point on the elliptic curve with an x- and a y-coordinate; think (0, 1), (42, 138), or (34876, 4893). None of these are public keys on our curve, though. But you get the idea.

To retrieve the public point from the private number, you conduct an elliptic curve sequence multiplication of the used base point G and the scalar (your private key). The result is another point, your public key.

The asymmetry of the Ethereum account cryptography lies in the ability to prove with the private number that you can access the public point. In contrast, knowledge about the public key never allows to reverse-reveal the private key.

Therefore, keeping the private key secret gives only you and nobody else control over the public key of the Ethereum account.

# Generate a secure random private-public
# keypair using the `eth` gem.
require "eth"
require "securerandom"
secret = SecureRandom.hex 32
# => "bec52dffb33ec1f4d629f88232796e898f4294079c5894a6645e8a4f7261fabe"
key = Eth::Key.new priv: secret
# => #<Eth::Key:0x000055ae60f86d58
#  @private_key=
#   #<Secp256k1::PrivateKey:0x000055ae60f86a38
#    @data=
#     "\xBE\xC5-\xFF\xB3>\xC1\xF4\xD6)\xF8\x822yn\x89\x8FB\x94\a\x9CX\x94\xA6d^\x8AOra\xFA\xBE">,
# @public_key=#<Secp256k1::PublicKey:0x000055ae60f869e8>>

# The private key is just a number.
key.private_key
# => #<Secp256k1::PrivateKey:0x000055ae60e81458
#  @data=
#   "\xBE\xC5-\xFF\xB3>\xC1\xF4\xD6)\xF8\x822yn\x89\x8FB\x94\a\x9CX\x94\xA6d^\x8AOra\xFA\xBE">
key.private_hex
# => "bec52dffb33ec1f4d629f88232796e898f4294079c5894a6645e8a4f7261fabe"

# The public key is a ... point?
key.public_key
# => #<Secp256k1::PublicKey:0x000055ae60e813e0>
key.public_hex
# => "040f9802cc197adf104916a6f94f6c93374647db7a3b774586ede221f1eea92b11e02a4be750aa0fe9cf975cec1b69a222841648d4c2ced7b1d108a2c9723e89b8"

Now, we used the eth gem to generate a keypair containing a private and a public key and we can easily see that bec52dffb33ec1f4d629f88232796e898f4294079c5894a6645e8a4f7261fabe is a number. It's the hexadecimal representation of the decimal number 86287827574830678407859947509786169732412250582090939460672560997304142789310. As I said, no magic. Notably, we choose such a huge number to prevent anyone guessing our key.

But what is the public key; it does not look like a point, right? That's because 040f9802cc197adf104916a6f94f6c93374647db7a3b774586ede221f1eea92b11e02a4be750aa0fe9cf975cec1b69a222841648d4c2ced7b1d108a2c9723e89b8 is a serialization of three fields: a prefix byte indicating the type of public key 04, the x-coordinate of the point 0f9802cc197adf104916a6f94f6c93374647db7a3b774586ede221f1eea92b11, and the y-coordinate e02a4be750aa0fe9cf975cec1b69a222841648d4c2ced7b1d108a2c9723e89b8.

That gives us the point (7053272788600477553676465022741516421197397404297740301327606102773230807825, 101392809526590995390445177899351250341338058145722255694397773992111072250296) on the elliptic curve. The prefix byte is not really used by Ethereum and only relevant for (un-)compressing keys in Bitcoin, so let's ignore it for the time being and assume it's always 04.

Now that we have a secret number 86287827574830678407859947509786169732412250582090939460672560997304142789310 representing the public point (7053272788600477553676465022741516421197397404297740301327606102773230807825, 101392809526590995390445177899351250341338058145722255694397773992111072250296), what's next? What can we do with this?

Addresses and Checksums

To recap, an account is a secret number giving access to a public point on an elliptic curve, easy as that! But now, we want to use some Ethereum blockchain functionality, such as transferring tokens or interacting with decentralized exchanges.

To receive Ether or any other asset on the Ethereum blockchain, you don't want to always tell your friends or family to send it to the coordinates (7053272788600477553676465022741516421197397404297740301327606102773230807825, 101392809526590995390445177899351250341338058145722255694397773992111072250296) which point to your spot on the curve where only you have access to. That would be slightly inconvenient.

Instead, we use addresses.

# The address of the previously 
# generated keypair.
key.address
# => #<Eth::Address:0x000055ae60fb8178
#  @address="0xc16fd2b4d06bcc9407b4b000b3085832f180f557">
key.address.to_s
# => "0xc16Fd2B4d06BCc9407b4B000b3085832F180F557"

And address is directly derived from your public key: you remove the prefix byte, do one round of Keccak-256 hashing, and take the last 20 bytes. That's it, that's the address - 20 hexadecimal bytes of the hash: 0xc16Fd2B4d06BCc9407b4B000b3085832F180F557.

require "digest/keccak"

# Pack the public key nicely into a byte string.
public_key = [key.public_hex].pack "H*"
# => "\x04\x0F\x98\x02\xCC\x19z\xDF\x10I\x16\xA6\xF9Ol\x937FG\xDBz;wE\x86\xED\xE2!\xF1\xEE\xA9+\x11\xE0*K\xE7P\xAA\x0F\xE9\xCF\x97\\\xEC\ei\xA2\"\x84\x16H\xD4\xC2\xCE\xD7\xB1\xD1\b\xA2\xC9r>\x89\xB8"

# Cut off the first prefix byte.
public_coordinates = public_key[1..-1]
# => "\x0F\x98\x02\xCC\x19z\xDF\x10I\x16\xA6\xF9Ol\x937FG\xDBz;wE\x86\xED\xE2!\xF1\xEE\xA9+\x11\xE0*K\xE7P\xAA\x0F\xE9\xCF\x97\\\xEC\ei\xA2\"\x84\x16H\xD4\xC2\xCE\xD7\xB1\xD1\b\xA2\xC9r>\x89\xB8"

# Hash the coordinate bytes of the pubic point
address_hash = Digest::Keccak.new(256).digest public_coordinates
# => "_s\xB6RA\xD0r\xF8x\xE2\xD1\xC2\xC1o\xD2\xB4\xD0k\xCC\x94\a\xB4\xB0\x00\xB3\bX2\xF1\x80\xF5W"

# Only grab the last 20 bytes.
address_bin = address_hash[-20..-1]
# => "\xC1o\xD2\xB4\xD0k\xCC\x94\a\xB4\xB0\x00\xB3\bX2\xF1\x80\xF5W"

# Unpack the address and prefix it with `0x`.
address = "0x#{address_bin.unpack("H*").first}"
# => "0xc16fd2b4d06bcc9407b4b000b3085832f180f557"

Note that by hashing the public key and cutting off the first 12 bytes of the hash, there is no way to go back and recover the public key from an address. The address is simply a placeholder on the blockchain for an account.

Only if you have a private key that maps to a public point that hashes to the exact address, solely, in this case, will the blockchain grant you access to any asset stored in the name of that placeholder on the ledger. How you can prove this cryptographically is beyond the scope of this article, and we might revisit this topic of signatures and transactions later.

One last little detail is left to investigate: Did you notice something unusual about the address? Right, key.address.to_s returned 0xc16Fd2B4d06BCc9407b4B000b3085832F180F557 which contains mixed-case letters. It's not only a 20-bytes hexadecimal string but also seems to feature random upper- and lower-case letters. Why is that?

This variance in the casing is the address checksum as per EIP-55: Mixed-case Checksum Address Encoding. To prevent typos or other mistakes while copying, pasting, or transmitting addresses, the hex-string has a checksum encoded on top of the address!

require "digest/keccak"

# Remove the address' hex-prefix.
unprefixed_address = address[2..-1]
# => "c16fd2b4d06bcc9407b4b000b3085832f180f557"

# Get the Keccak-256 hash of the 
# unprefixed address.
checksum = Digest::Keccak.new(256).digest(unprefixed_address.downcase).unpack("H*").first
# => "4bd92ec1770ff46b882ff0297df0ab4ee199a0b1947d3a378089e7127ca58d60"

# Map the checksum to address chars and 
# determine capitalization.
checksummed_chars = unprefixed_address.chars.zip(checksum.chars).map do |addr, chck|
  chck.match(/[0-7]/) ? addr.downcase : addr.upcase
end
# => ["c", "1", "6", "F", "d", "2", "B", "4", "d", "0", "6", "B", "C", "c", "9", "4", "0", "7", "b", "4", "B", "0", "0", "0", "b", "3", "0", "8", "5", "8", "3", "2", "F", "1", "8", "0", "F", "5", "5", "7"]

# Et voilà, une addresse.
checksummed_address = "0x#{checksummed_chars.join}"
# => "0xc16Fd2B4d06BCc9407b4B000b3085832F180F557"

The checksum algorithm hashes the address again with one round of Keccak-256 and treats this hash as the checksum. Then, for each character in the address, we check if the corresponding hex digit in the checksum is either 0..7 or 8..f. If it's less than 8, we encode a lower-case letter; otherwise, we use an upper-case letter. That's it.

Smart-Contract Accounts

We understand that an address is just a placeholder on the blockchain for storing tokens or any other arbitrary asset that can be unlocked only by a particular private number.

But who controls the accounts of smart contracts? What is a smart-contract account in the first place?

A smart contract is, plainly speaking, just executable code. This code gets deployed to a smart-contract account, an address, a placeholder on the blockchain, where the private key is unknown.

This is intentional. If the secret giving access to a smart-contract account was known, you would never be able to guarantee that such a contract cannot be tampered with.

To determine a smart-contract address, you need two things: the sender account address and nonce while deploying the contract.

require "eth"
require "digest/keccak"

# Our sender address, an externally 
# owned account.
sender = "0xc16Fd2B4d06BCc9407b4B000b3085832F180F557"
# => "0xc16Fd2B4d06BCc9407b4B000b3085832F180F557"

# Our sender's nonce is `0` because we never 
# used this account before.
nonce = 0
# => 0

# RLP-encode both sender and nonce.
encoded = Eth::Rlp.encode [sender, nonce]
# => "\xEC\xAA0xc16Fd2B4d06BCc9407b4B000b3085832F180F557\x80"

# Apply one round of Keccak-256 hashing.
hashed = Digest::Keccak.new(256).digest encoded
# => ">0\x1D\xB1\xA4Lv\x04\xD3ul\x10\xA2sT\xDA\xD4\x9C]\x9C\r\x9A +d\xD6\x80\xC5\xFCN\xFC|"

# Again, only get the last 20 bytes 
# of the hash.
address = hashed[-20..-1].unpack("H*").first
# => "a27354dad49c5d9c0d9a202b64d680c5fc4efc7c"

# And that's the address.
address = Eth::Address.new("0x#{address}").to_s
# => "0xa27354dAd49c5d9C0D9a202b64D680c5fC4efC7C"

The sender address and nonce are put into an array and RLP-encoded. RLP is Ethereum's Recursive Length Prefix serializer. The encoded RLP is then again hashed with Keccak-256, and the last 20 bytes of the hash are the expected contract account address.

Note that because we don't hash a public key but instead an RLP-encoded object, there is a certainty that we can not know the private key for the smart-contract account 0xa27354dAd49c5d9C0D9a202b64D680c5fC4efC7C.

What did I just read?

To recap, an Ethereum account is, just like many other cryptographic accounts (OTR, PGP, SSH), a private-public keypair:

the private key is just a huge number, with 32 bytes of entropy.
the public key is a corresponding point with x- and y-coordinates on the underlying elliptic curve (here: SECP256K1, same as Bitcoin uses).
an Ethereum address is a truncated Keccak-256 hash of the serialized public key.
the Ethereum address checksum is encoded by mixed-case letters in the address (EIP-55).
no one owns smart-contract accounts, and their private keys are unknown on purpose.

Let me know if this makes sense in the comments below!

Further Resources

q9f/eth.rb: A Ruby library to build, sign, and broadcast Ethereum transactions.
q9f/keccak.rb: Ruby bindings for the Keccak hash (non-final SHA-3) used by Ethereum.
q9f/secp256k1.cr: An educational library implementing Secp256k1 well readable, purely for the Crystal language.

Ethereum 2.0 Mainnet Clients

Afri — Tue, 02 Feb 2021 18:07:15 +0000

Comparison of all available Ethereum 2.0 (Eth2) mainnet clients based on latest performance metrics.

After the Ethereum 2.0 mainnet launch of the beacon chain in December 2020, it is just about time to introduce and compare the existing protocol implementations. The first part of this mini-series of articles will compare the beacon-node performance and resource utilization of the five main clients, in alphabetical order:

Lighthouse (Rust, Sigma Prime)
Lodestar (TypeScript, ChainSafe Systems)
Nimbus (Nim, Status)
Prysm (Go, Prysmatic Labs)
Teku (Java, ConsenSys Quorum)

The Ethereum 2.0 mainnet infrastructure consists of three major components:

The beacon chain is a proof-of-stake blockchain. In the future, this will be the backbone of keeping Ethereum secure once the merge of the legacy proof-of-work Ethereum 1.x chain is concluded.
The validators are the miners in proof-of-stake. By locking 32 ETH, everyone can stake their Ether, propose new blocks, vote on finality, and obtain rewards.
The slashers are monitoring the validators for their correct behavior to prevent attacks. In case any validator breaks the rules, they will be penalized and removed from the network.

Notably, in this article, we focus on the first element. The beacon chain is the foundation for all other components in Ethereum 2. Researchers can find all relevant scripts, data, and plots on Github for further analysis.

byz-f / eth2-bench-mainnet

This article focuses on describing the findings.

Synchronization Metrics

The first and potentially most exciting question is how long it takes to synchronize the Ethereum 2.0 beacon chain. Well, here are the results.

You can read the synchronization progress in slot numbers over time of the client running. Before nominating a winner, which is not the scope of this article, there are three things to know about this chart.

Prysm (purple) does something unique that the other clients do not do anymore. It connects to an Ethereum 1.x node, fetches all Ether deposits from the validator registry, and builds the Eth2 genesis from the Eth1 state. While this is a useful feature from a security perspective, as you do not have to trust the Prysm developers to give you the correct genesis state, this takes time. Therefore, there is a visible offset between client start and synchronization start (#8209).
Lodestar (gray) crashed during the benchmark due to a JavaScript heap out-of-memory issue (#2005). However, it was automatically restarted by the scripts after 10 seconds.
Not visible: Lodestar does not yet fully validate all signatures on initial sync (#1217). Therefore, it is not clear how the Lodestar graph compares to the others.

Given this chart, Lighthouse (orange) shows an outstanding overall out-of-the-box performance, with Prysm, Teku (green), and Nimbus (blue) doing an excellent job at keeping up. But, let's also take a look at this.

In this chart, we removed the time offset between launching the client and the client's first beacon chain block during synchronization. You can see that Prysm beats Lighthouse in pure synchronization speed slightly, requiring a little less than two hours, where Lighthouse requires two and a half hours. Teku and Nimbus are still showing good performance with roughly five hours required.

Notably, the Eth2 TypeScript implementation's strength is not being the go-to client for running a full beacon-chain or validator node. Instead, Lodestar provides the infrastructure for all web-, browser-, and plugin-based components of Eth2 decentralized applications in the future.

Given we know the current slot height of the client's beacon head block and we can look up the head it saw 60 seconds ago, we can compute the synchronization speed as moving average over the last 60 seconds displayed at slots per second (dots). A moving average over 10 minutes is expressed as a solid line.

The results reflect the previous charts. Prysm, even with the Eth1-state offset, is the fastest client synchronizing 60 slots per second, closely followed by Lighthouse with 46 slots per second. Slightly behind in the field are Teku with 23 and Nimbus with 22 slots per second.

But, what is a slot, you may ask. There is either a block in traditional blockchains, such as Bitcoin and Eth1, or there is not. To compute the performance of such clients, we would compare the synchronization speed in blocks per second. What's the difference?

In Ethereum 2.0, there is always an assigned slot scheduled at a fixed interval; here: 12 seconds. If the validator assigned to such a slot proposes a block, we see a block in that slot. However, if a validator misses its slot, the slot will be empty (no block), but the slot count will move on regardless. Therefore, in Eth2, we compare the sync-speed in slots per second.

Let's remove the time component from the chart above and solely focus on synchronization speed mapped over the synchronized slot number. Across all clients, there is a trend visible that sync performance decreases over slot number. As this data was gathered on the Ethereum 2.0 mainnet, we know there is a queue of validators waiting to join the network. At the time of writing, 13_458 validators are in the queue, which would take almost 15 days to clear at a rate of 900 new validators every day.

Knowing about the linear growth of the Eth2 mainnet validator count, we can assume that the active validator set's size negatively impacts the synchronization speed.

Computational Resource Metrics

In the previous section, we solely analyzed synchronization metrics to find the fastest client. But which client is not only fast but also efficient in their resource utilization?

Here, the size of the database is compared across the five clients to the synchronized slot number. Notably, Lodestar comes with the smallest footprint totaling only 1.49 GiB for the fully synchronized mainnet node (420_000 slots). Lighthouse (2.98 GiB) and Prysm (3.16 GiB) follow with good results.

We know that Eth1 nodes store the full block history. Still, they remove historical states to minimize the disk space required for their databases. Eth2 nodes are reasonably comparable to this concept. While keeping all blocks on disk, they remove finalized states. The main difference here is that it's worth keeping historical states on epoch boundaries for convenience. Currently, Nimbus keeps the state on every 32nd epoch boundary while Lodestar only writes the state of every 1024th epoch to disk. The differences are well visible in the plot.

This chart is the same but plotting the resident memory set size of each client during synchronization. Here, the Nimbus client is highly efficient, requiring only about 1 GiB of RAM during the entire processing of the beacon-chain mainnet. Nimbus is followed by Lighthouse and Lodestar, both operating at slightly less than 3 GiB.

One shall note that the off-heap memory that Java allocates for Teku is outside of the client developers' control. The JVM is greedy about available memory. The results of these metrics for Teku vary massively based on the total memory available.

Last but not least, let's take a look at the CPU utilization. Here, we can also see some interesting differences between the clients.

A blockchain is a highly hierarchical data-structure. Most of the task for synchronizing the chain, validating the blocks, and computing the latest states is a reasonably sequential job. So, the challenge for the clients is to parallelize this process as good as possible. The results above are comparable to the sync speed metrics, with Prysm and Lighthouse leading the field (higher is more efficient) and Teku keeping well up.

That's it for now! Any questions?

FAQ

Excellent article, but why didn't you compare traffic metrics?

I did; however, I didn't comment on them all. You can find all plots, including peering and traffic metrics unannotated on Github for a further deep dive: eth2-bench-mainnet/doc/00-plots-uncommented.md

Which client do you personally recommend?

That's a tough question. My gut feeling would be leaning towards Lighthouse with the best overall experience, good performance, and many features and tooling available. Still, I would also always run Prysm as it is the most mature and currently the fastest client. My overall experience with Teku is also very satisfying, and I would consider all of them production-ready.

While I have your attention, will the beacon chain ever be more than 1 TiB?

No, comparable to Eth1, the beacon chain in itself is relatively small. The primary factor that could be driving the database size is the beacon state. However, also comparable to Eth1, it's not required to save all states to disk as you would be able to always reconstruct any state from the blocks that you have locally.

In addition to that, proof-of-stake offers finality, which proof-of-work doesn't (reorgs, 51%-attacks). After a certain period, a block is declared final, which means it can never be changed again. The finality concept would allow clients in the future, instead of syncing the chain from genesis, to fetch the latest chain head since the last finalized epoch.

Haven't you done that before, or anyone else?

Yes, on Medalla testnet last year:

q9f / eth2-bench-2020-10

Also, read the Resource Analysis of Ethereum 2.0 Clients conducted by the Barcelona Supercomputing Center.

Thank you for scrolling down!

How to run your own Beacon Chain

Afri — Thu, 19 Mar 2020 12:07:06 +0000

This article is for educational purposes only. A beacon chain is a future Ethereum 2.0 relay connecting shard chains. There's literally no value in running a beacon chain currently as there are no shards yet and there's no other functionality on the beacon chain available. But it's new technology, and who's not excited about playing around with it?

To bootstrap a new, custom beacon chain, we would need to cover the following topics:

The deposit contract ceremony
The beacon chain transition specification
The genesis event
Bootstrapping the actual network

Before we start, we need to understand that the beacon chain, which we will also call eth2 chain for now, cannot live without a legacy Ethereum chain which we will call eth1 chain. There will be a transition from eth1 to eth2 defined by an interface, the so called deposit contract.

The deposit contract ceremony

The deposit contract is a tool that allows gathering Ether deposits on the eth1 chain to enable validators proposing blocks on the eth2 chain later. How this will work in detail, will be covered later. For now, we need to understand that we need the deposit contract before doing anything else.

An example of the deposit contract is available as validator_registration.vy in the eth2.0-specs repository written in the Vyper contract language. The main function of interest of that fairly simple contract is -- surprise -- deposit(). But we'll cover that later.

We need three things now:

The compiled bytecode of the contract
The ABI of the contract
And an eth1 testnet to deploy it to

To compile the contract, we need to install Vyper locally or use an Vyper Online Compiler. The online-compiler comes in pretty handy in some cases, but for such an important piece of infrastructure, we'll use the native compiler locally. It's simple as that:

❯ vyper validator_registration.vy
0x740100000000000000000000000000000000000000006020526f7fffffffffffffffffffffffffffffff6040527fffffffffffffffffffffffffffffffff8000000000000000000000000000000060605274012a05f1fffffffffffffffffffffffffdabf41c006080527ffffffffffffffffffffffffed5fa0e000000000000000000000000000000000060a052341561009857600080fd5b6101406000601f818352015b600061014051602081106100b757600080fd5b600260c052602060c020015460208261016001015260208101905061014051602081106100e357600080fd5b600260c052602060c020015460208261016001015260208101905080610160526101609050602060c0825160208401600060025af161012157600080fd5b60c05190506101405160016060518183018060405190131561014257600080fd5b809190121561015057600080fd5b905090506020811061016157600080fd5b600260c052602060c02001555b81516001018083528114156100a4575b50506110a256600436101561000d57610f17565b600035601c52740100000000000000000000000000000000000000006020526f7fffffffffffffffffffffffffffffff6040527fffffffffffffffffffffffffffffffff8000000000000000000000000000000060605274012a05f1fffffffffffffffffffffffffdabf41c006080527ffffffffffffffffffffffffed5fa0e000000000000000000000000000000000060a05260001561020e575b6101605261014052600061018052610140516101a0526101c060006008818352015b61018051600881811b90509050610180526101805160ff6101a051168181830110156100f657600080fd5b80820190509050610180526101a0517ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff88160081c905090506101a0525b81516001018083528114156100cb575b505060186008602082066101e0016020828401111561016157600080fd5b60208061020082610180600060045af15050818152809050905090508051602001806102a08284600060045af161019757600080fd5b50506102a051806102c001818260206001820306601f820103905003368237505060206102805260406102a0510160206001820306601f8201039050610260525b6000610260511115156101ea57610206565b60206102605103610280015160206102605103610260526101d8565b610160515650005b63c5f2892f60005114156104a257341561022757600080fd5b6000610140526101405161016052600154610180526101a060006020818352015b60016001610180511614156102c95760006101a0516020811061026a57600080fd5b600060c052602060c02001546020826102400101526020810190506101605160208261024001015260208101905080610240526102409050602060c0825160208401600060025af16102bb57600080fd5b60c051905061016052610337565b6000610160516020826101c00101526020810190506101a051602081106102ef57600080fd5b600260c052602060c02001546020826101c0010152602081019050806101c0526101c09050602060c0825160208401600060025af161032d57600080fd5b60c0519050610160525b61018080516002808061034957600080fd5b8204905090508152505b8151600101808352811415610248575b505060006101605160208261046001015260208101905061014051610160516101805163806732896102e0526001546103005261030051600658016100a9565b506103605260006103c0525b6103605160206001820306601f82010390506103c0511015156103d1576103ea565b6103c05161038001526103c0516020016103c0526103af565b61018052610160526101405261036060088060208461046001018260208501600060045af150508051820191505060006018602082066103e0016020828401111561043457600080fd5b60208061040082610140600060045af150508181528090509050905060188060208461046001018260208501600060045af150508051820191505080610460526104609050602060c0825160208401600060025af161049257600080fd5b60c051905060005260206000f350005b63621fd130600051141561057f5734156104bb57600080fd5b6380673289610140526001546101605261016051600658016100a9565b506101c0526000610220525b6101c05160206001820306601f8201039050610220511015156105065761051f565b610220516101e0015261022051602001610220526104e4565b6101c08051602001806102808284600060045af161053c57600080fd5b505061028051806102a001818260206001820306601f82010390500336823750506020610260526040610280510160206001820306601f8201039050610260f350005b63228951186000511415610f165760506004356004016101403760306004356004013511156105ad57600080fd5b60406024356004016101c03760206024356004013511156105cd57600080fd5b60806044356004016102203760606044356004013511156105ed57600080fd5b63ffffffff600154106105ff57600080fd5b34633b9aca00808061061057600080fd5b8204905090506102c052633b9aca006102c051101561062e57600080fd5b6030610140511461063e57600080fd5b60206101c0511461064e57600080fd5b6060610220511461065e57600080fd5b610140610340525b6103405151602061034051016103405261034061034051101561068857610666565b6380673289610360526102c0516103805261038051600658016100a9565b506103e0526000610440525b6103e05160206001820306601f8201039050610440511015156106d4576106ed565b61044051610400015261044051602001610440526106b2565b610320610340525b6103405152602061034051036103405261014061034051101515610718576106f5565b6103e08051602001806102e08284600060045af161073557600080fd5b5050610140610460525b610460515160206104605101610460526104606104605110156107615761073f565b6380673289610480526001546104a0526104a051600658016100a9565b50610500526000610560525b6105005160206001820306601f8201039050610560511015156107ac576107c5565b610560516105200152610560516020016105605261078a565b610440610460525b61046051526020610460510361046052610140610460511015156107f0576107cd565b6105008051602001806105808284600060045af161080d57600080fd5b505060a06105e0526105e051610620526101408051602001806105e051610620018284600060045af161083f57600080fd5b50506105e05161062001518060206105e051610620010101818260206001820306601f820103905003368237505060206105e051610620015160206001820306601f82010390506105e05101016105e0526105e051610640526101c08051602001806105e051610620018284600060045af16108ba57600080fd5b50506105e05161062001518060206105e051610620010101818260206001820306601f820103905003368237505060206105e051610620015160206001820306601f82010390506105e05101016105e0526105e051610660526102e08051602001806105e051610620018284600060045af161093557600080fd5b50506105e05161062001518060206105e051610620010101818260206001820306601f820103905003368237505060206105e051610620015160206001820306601f82010390506105e05101016105e0526105e051610680526102208051602001806105e051610620018284600060045af16109b057600080fd5b50506105e05161062001518060206105e051610620010101818260206001820306601f820103905003368237505060206105e051610620015160206001820306601f82010390506105e05101016105e0526105e0516106a0526105808051602001806105e051610620018284600060045af1610a2b57600080fd5b50506105e05161062001518060206105e051610620010101818260206001820306601f820103905003368237505060206105e051610620015160206001820306601f82010390506105e05101016105e0527f649bbc62d0e31342afea4e5cd82d4049e7e1ee912fc0889aa790803be39038c56105e051610620a160006106c052600061014060308060208461078001018260208501600060045af150508051820191505060006010602082066107000160208284011115610aeb57600080fd5b602080610720826106c0600060045af150508181528090509050905060108060208461078001018260208501600060045af150508051820191505080610780526107809050602060c0825160208401600060025af1610b4957600080fd5b60c05190506106e0526000600060406020820661082001610220518284011115610b7257600080fd5b6060806108408260206020880688030161022001600060045af1505081815280905090509050602060c0825160208401600060025af1610bb157600080fd5b60c0519050602082610a20010152602081019050600060406020602082066108e001610220518284011115610be557600080fd5b6060806109008260206020880688030161022001600060045af15050818152809050905090506020806020846109a001018260208501600060045af15050805182019150506106c0516020826109a0010152602081019050806109a0526109a09050602060c0825160208401600060025af1610c6057600080fd5b60c0519050602082610a2001015260208101905080610a2052610a209050602060c0825160208401600060025af1610c9757600080fd5b60c051905061080052600060006106e051602082610ac00101526020810190506101c0602080602084610ac001018260208501600060045af150508051820191505080610ac052610ac09050602060c0825160208401600060025af1610cfc57600080fd5b60c0519050602082610c4001015260208101905060006102e0600880602084610bc001018260208501600060045af15050805182019150506000601860208206610b400160208284011115610d5057600080fd5b602080610b60826106c0600060045af1505081815280905090509050601880602084610bc001018260208501600060045af150508051820191505061080051602082610bc001015260208101905080610bc052610bc09050602060c0825160208401600060025af1610dc157600080fd5b60c0519050602082610c4001015260208101905080610c4052610c409050602060c0825160208401600060025af1610df857600080fd5b60c0519050610aa052606435610aa05114610e1257600080fd5b600180546001818183011015610e2757600080fd5b80820190509050815550600154610cc052610ce060006020818352015b60016001610cc051161415610e7957610aa051610ce05160208110610e6857600080fd5b600060c052602060c0200155610f12565b6000610ce05160208110610e8c57600080fd5b600060c052602060c0200154602082610d00010152602081019050610aa051602082610d0001015260208101905080610d0052610d009050602060c0825160208401600060025af1610edd57600080fd5b60c0519050610aa052610cc0805160028080610ef857600080fd5b8204905090508152505b8151600101808352811415610e44575b5050005b5b60006000fd5b6101856110a2036101856000396101856110a2036000f3

This is our compiled bytecode which can be deployed later. Now we need the ABI:

❯ vyper -f abi validator_registration.vy
[{"name": "DepositEvent", "inputs": [{"type": "bytes", "name": "pubkey", "indexed": false}, {"type": "bytes", "name": "withdrawal_credentials", "indexed": false}, {"type": "bytes", "name": "amount", "indexed": false}, {"type": "bytes", "name": "signature", "indexed": false}, {"type": "bytes", "name": "index", "indexed": false}], "anonymous": false, "type": "event"}, {"outputs": [], "inputs": [], "constant": false, "payable": false, "type": "constructor"}, {"name": "get_deposit_root", "outputs": [{"type": "bytes32", "name": ""}], "inputs": [], "constant": true, "payable": false, "type": "function", "gas": 112305}, {"name": "get_deposit_count", "outputs": [{"type": "bytes", "name": ""}], "inputs": [], "constant": true, "payable": false, "type": "function", "gas": 14558}, {"name": "deposit", "outputs": [], "inputs": [{"type": "bytes", "name": "pubkey"}, {"type": "bytes", "name": "withdrawal_credentials"}, {"type": "bytes", "name": "signature"}, {"type": "bytes32", "name": "deposit_data_root"}], "constant": false, "payable": true, "type": "function", "gas": 1331065}]

The ABI will allow us later to interact with the deposit contract once it is deployed. As testnet, we will use the Görli Testnet. We will go on straight deploying the bytecode from above and have a contract address: 0xaA888248144Bc5d584a7f400839d0D912F21C39A.

That's it, for a testnet at least. For a mainnet, this process will involve an actual ceremony to determine social consensus on the contract code and the official instance to use.

For the sake of easing eth2 clients with eth1 synchronization, we also note down two values:

The deposit contract deploy block: 2355031
The deposit contract deploy date: 1584316800 (March/16, 2020)

The beacon chain transition specification

Now that we have an instance of the deposit contract deployed to a public eth1 network, we'll start configuring our eth2 network transitions. For that, we can template our future network on the existing specification for the official Ethereum 2.0 mainnet, namely mainnet.yaml.

We'll take that specification and modify it slightly to ease running a testnet with just a couple of validators:

DEPOSIT_CONTRACT_ADDRESS: 0xaA888248144Bc5d584a7f400839d0D912F21C39A - We deployed this contract in the previous step.
GENESIS_FORK_VERSION: 0x00000539 - We just change this to something that's not 0x0 to distinguish our network from mainnet.
BLS_WITHDRAWAL_PREFIX: 0x05 - We change this to something that's not 0x0 to not conflict with mainnet.
MIN_GENESIS_ACTIVE_VALIDATOR_COUNT: 16 - We want to be able to bootstrap the network with as little as 16 validators.
MIN_GENESIS_TIME: 1584316800 - This is our day we deployed the deposit contract, the genesis cannot happen before (Mar/16, 2020).
MIN_DEPOSIT_AMOUNT: 100000000 - We reduce the minimum deposit amount to 0.1 GöETH.
MAX_EFFECTIVE_BALANCE: 3200000000 - We reduce the maximum effective balance to 3.2 GöETH.
EJECTION_BALANCE: 1600000000 - We reduce the ejection balance to 1.6 GöETH.
EFFECTIVE_BALANCE_INCREMENT: 100000000 - We reduce the effective balance increment to 0.1 GöETH.
ETH1_FOLLOW_DISTANCE: 16 - We change the eth1 follow distance to 16.
MIN_GENESIS_DELAY: 3600 - We reduce the minimum genesis delay to one hour.

That's it, we just fully configured our eth2 network. I'll call this network Schlesi because it's a subway station close to Görli and it will be easier to reference it by a name. I compiled all configurations we did so far in a Gist on Github which can be feeded to clients later.

The genesis event

After fully configuring the network, we still don't have a network, yet. What's missing is the genesis state. The genesis is determined by a certain starting condition which is defined in the clients and by our previous network specification. It mainly breaks down to the following criteria:

The minimum number of active validator deposits: We configured 16 and the genesis event will not happen before the deposit contract registered 16 valid deposits.
The minimum genesis time: We configured March/16 2020 and the genesis event will not be happening before that date.
The minimum genesis delay: We configured to delay the genesis state by at least one hour. The genesis will not happen before we don't have 16 validator deposits and one hour passed.

Once these conditions are met, an eth2 client will be able to extract the genesis state from the eth1 chain and validators can start proposing blocks.

The Lighthouse client not only comes with beacon chain and a validator client, but also with a command line tool lcli that assists us in generating keypairs for validators and sending the required validator deposit directly to the deposit contract on the eth1 chain.

First, we instruct our lighthouse node how to configure the new testnet:

lcli --spec mainnet new-testnet \ # template new testnet on mainnet
--deposit-contract-address 0xaA888248144Bc5d584a7f400839d0D912F21C39A \
--deposit-contract-deploy-block 2355031 \
--effective-balance-increment 100000000 \
--ejection-balance 1600000000 \
--eth1-follow-distance 16 \
--genesis-fork-version 0x00000539 \
--max-effective-balance 3200000000 \
--min-deposit-amount 100000000 \
--min-genesis-active-validator-count 16 \
--min-genesis-delay 3600 \
--min-genesis-time 1584316800 \
--testnet-dir ~/.lighthouse/schlesi

These are the values we defined in the previous step and have now been written to our testnet directory ~/.lighthouse/schlesi for further use by the Lighthouse client. To run a beacon node, simply point it to the config:

lighthouse bn \ # lighthouse beacon node
--testnet-dir ~/.lighthouse/schlesi \
--spec mainnet \
--eth1-endpoint http://127.0.0.1:8545 \
--http

Now, our beacon node monitors our local Görli node on localhost port :8545 and waits that something happens on the deposit contract.

We can use lighthouse now to make validator deposits:

lighthouse account validator new \
--testnet-dir ~/.lighthouse/schlesi \
--spec mainnet \
--send-deposits \
--eth1-endpoint http://127.0.0.1:8545 \
--password ./password.txt \
--deposit-value 3200000000 random

This will do the following things:

It reads our testnet spec in ~/.lighthouse/schlesi mainly to know about the deposit contract.
It generates a keypair for a validator in ~/.lighthouse/validators.
It generates a transaction and broadcasts it to the eth1 network using your local node. Important to note here, is that you have enough testnet Ether on the first account of your local Görli node (eth.accounts[0]) and that you provide a valid password to unlock that account.

This does not work with Parity Ethereum, so we used Geth instead. Also, to have both, HTTP-RPC APIs exposed and being able to unlock the accounts, one would have to pass the --allow-insecure-unlock flag to the Görli node. Please, use this with caution!

Once this was done, the node will give positive feedback:

Mar 18 13:53:48.174 INFO Validator deposit successful            validator_voting_pubkey: 0x82ad6def329bf94f06dae46b3222a3cb9baf2a68e1bf6e1dadaac5bb47dfe6690e2c3222b6256e17a4ec1bdbec6dbd65, eth1_tx_hash: 0x13cb9c913d946b222cb595f4bb42bd3e7d11e985d1d4dc3c50faacdc8e42e053

This is how a deposit transaction to our contract looks like. Repeat this step 16 times to meet the minimum genesis deposit threshold. The beacon chain closely monitoring the deposit contract notices the met threshold:

Mar 18 13:55:42.991 INFO Minimum genesis deposit count met       block_number: 2366961, deposit_count: 16, service: beacon
Mar 18 13:55:57.647 INFO Deposit contract genesis complete       validator_count: 16, eth1_block_height: 2366961, service: beacon
Mar 18 13:55:57.680 INFO Beacon chain initialized                head_slot: 0, head_block: 0xd0ad…1b0d, head_state: 0xdadf…c6bc, service: beacon

The genesis state can be extracted from the eth1 contract using lcli:

lcli eth1-genesis \
--testnet-dir ~/.lighthouse/schlesi \
--eth1-endpoint http://127.0.0.1:8545

Bootstrapping the actual network

Now that the genesis event happened, we need two last things to bootstrap the actual network:

Validators proposing new blocks
Bootnode records allowing for other nodes to connect

Fortunately, both is available in Lighthouse. To run your 16 validators configured locally, simply run:

lighthouse vc --allow-unsynced

The --allow-unsynced option is required as we do not have any other peers yet and simply allow the validator to start proposing blocks right away.

Mar 18 13:57:20.871 WARN Ethereum 2.0 is pre-release. This software is experimental.
Mar 18 13:57:20.876 INFO Starting validator client               datadir: "/home/user/.lighthouse/validators", beacon_node: http://localhost:5052/
Mar 18 13:57:20.893 INFO Connected to beacon node                version: Lighthouse/v0.1.0-unstable/x86_64-linux
Mar 18 13:57:20.900 INFO Starting node prior to genesis          seconds_to_wait: 3759

The validator node now waits at least one hour before proposing blocks as defined in our MIN_GENESIS_DELAY.

Meanwhile, we can launch more beacon nodes and start wiring them. To connect them, we take the Ethereum Node Record (ENR) from each of the nodes and add them to our testnet configuration. The ENR looks as follows:

Mar 18 13:55:57.692 INFO ENR Initialised                         seq: 1, enr: enr:-Iu4QN6QRhX7abcUZ6E5eV9-AIUSXpNBSytiSNwG0FGWWifEevi98EDGhoPVt3e82j9KC2H7DiLtDGnj03MMrs707fEBgmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQL4Lu3nwG5diXILEh3LiAauCrNgtoTJxGZQER33BTDMKoN0Y3CCIyiDdWRwgiMo, service: network

Our testnet configuration in ~/.lighthouse/schlesi contains a boot_enr.yaml file which we use to collect ENR of known peers to ease discovery. We simply add all known ENR of our beacon-bootnodes to that file which allows the network to have a first point of entry when adding new beacon nodes. The file could look like that:

- "enr:-Iu4QGCUN3RjOLZCab4LLqlOqnnOB9BspIE30Nj5gQGoY00ZXCIW2PCXuGqDLHEPZJK9NO8SFZlKzFF-5rSbTyPqksoBgmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQJmy_5ZkaIFozyMd3gPygG1lVLQONuTL4C1j_smkZ9WmoN0Y3CCIyiDdWRwgiMo"
- "enr:-Iu4QCKxIZVqkBMGHOxfeDvY8Yp0V0uq2MQ8wFS2tQGMfQ1YVuER_WeyVmqKawz6H4JE1OVeN52_kJUdZmkuvpyETjMBgmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQNsaKrbnhjMgAcpHREHrZiCA7sXyEOokOck_1oc3zZMG4N0Y3CCJRyDdWRwgiUc"
- "enr:-Iu4QN6QRhX7abcUZ6E5eV9-AIUSXpNBSytiSNwG0FGWWifEevi98EDGhoPVt3e82j9KC2H7DiLtDGnj03MMrs707fEBgmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQL4Lu3nwG5diXILEh3LiAauCrNgtoTJxGZQER33BTDMKoN0Y3CCIyiDdWRwgiMo"

That's it. We have our own, custom beacon chain now running on Lighthouse.

Mar 21 16:38:26.000 INFO Synced                                  slot: 425, epoch: 13, finalized_epoch: 11, finalized_root: 0x41fc…1ac0, peers: 3, service: slot_notifier
Mar 21 16:38:32.226 INFO Block from local validator              block_slot: 426, block_root: 0xa2ef…e552, service: http
Mar 21 16:38:38.000 INFO Synced                                  slot: 426, epoch: 13, finalized_epoch: 11, finalized_root: 0x41fc…1ac0, peers: 3, service: slot_notifier
Mar 21 16:38:44.218 INFO Block from local validator              block_slot: 427, block_root: 0x563c…b946, service: http
Mar 21 16:38:48.018 INFO Attestation from local validator        slot: 427, index: 0, source: 12, target: 12, service: http
Mar 21 16:38:50.000 INFO Synced                                  slot: 427, epoch: 13, finalized_epoch: 11, finalized_root: 0x41fc…1ac0, peers: 3, service: slot_notifier
Mar 21 16:38:56.249 INFO Block from local validator              block_slot: 428, block_root: 0x8229…eb17, service: http
Mar 21 16:39:02.000 INFO Synced                                  slot: 428, epoch: 13, finalized_epoch: 11, finalized_root: 0x41fc…1ac0, peers: 3, service: slot_notifier
Mar 21 16:39:08.237 INFO Block from local validator              block_slot: 429, block_root: 0x4793…633b, service: http
Mar 21 16:39:12.012 INFO Attestation from local validator        slot: 429, index: 0, source: 12, target: 12, service: http
Mar 21 16:39:14.002 INFO Synced                                  slot: 429, epoch: 13, finalized_epoch: 11, finalized_root: 0x41fc…1ac0, peers: 3, service: slot_notifier
Mar 21 16:39:20.232 INFO Block from local validator              block_slot: 430, block_root: 0xabc4…4fb4, service: http

It's important to note, that this network is permissionless. Everyone with 3.2 Görli ETH can generate a validator keypair and make a deposit to the Schlesi validator registration contract. Everyone can be part running validator nodes or beacon chain nodes by simply connecting to the provided ENR. Why don't you give it a try?

Soon, hopefully, we will be able to add other clients to our network, such as Prysm, Nimbus, Teku, or Trinity. The multi-client testnet phase is soon to be upon us. 🙏

Happy Görli Testnet Launch Anniversary

Afri — Fri, 31 Jan 2020 06:11:30 +0000

On January 31, 2019, at 4:10:53 pm local time in Berlin, at the first Testnet Conference ever, GörliCon0, the Görli Testnet Block #1 was sealed. Let’s look at what we’ve achieved since our project was conceived and reflect. Happy Görli Testnet Launch Anniversary!

The Görli project has come a long way since it was first ideated in Toronto in Summer 2018 - and began development at ETHBerlin shortly afterward. Today we want to reflect on the achievements we made and the lessons we learned over the last 18 months.

Our idea was simple: an Ethereum testnet for application developers to work freely on, regardless of which tooling or client they use, and as reliable and stable as the main network. The existing testnets available back then were either bound to a certain tech stack (Rinkeby, Kovan) or too unstable (Ropsten) for application testing.

Therefore, at ETHBerlin 2018, some of the people from the ChainSafe team and Afri Schoedon took the first steps toward developing Görli and tried to specify the Authority Round (Aura) consensus engine that was powering Kovan and subsequently implementing it in Go for the Geth client. Long story short, the team failed to specify and implement it due to the unprecedented complexity of the engine; however, the idea was favored by the judges and the Görli project eventually won and was able to demo the project on stage.

That was probably the most important milestone of the project: Get the idea out of the door and gather feedback. The hackathon submission was unfortunately disqualified after other attendees fairly pointed out that we were much more people on stage than the rules allowed. We were so excited about this promising idea, that we accidentally bypassed the hackathon’s rules including “forcing” mentors to code - Thank you, by the way! In spite of the disqualification, the Department of Decentralization pledged to support Görli in any way possible - except for the prize money, of course, which was given to the runner up. We all understood this project had become unstoppable and decided to make it come to life.

The lesson learned at ETHBerlin 2018 - besides honoring hackathon rules - is that Aura is too complex to implement across a variety of clients and there are many unspecified features of that protocol that only exist within the code-base of the Parity Ethereum client. The EIP Draft for Aura is still available on Github if anyone bothers to pick this up one day, and so is Geth Aura.

It then became clear that we needed to investigate Geth's Clique engine and implement it in Rust. Fortunately, the Görli project was able to secure grants both from the ETC Cooperative in 2018 and further along the line, the Ethereum Foundation in 2019 that helped to realize the project on a broader scale. Many thanks to both organizations for their financial support.

To continue Görli’s development, we decided to pay individual contributors via Gitcoin bounties. Thanks to the hands-on work from Jared Wasinger and Yucong Sun pioneering the Parity Görli client, Görli was soon able to sync the Rinkeby testnet. This is when we decided to launch the Kotti Classic cross-client proof-of-authority testnet for Ethereum Classic in December 2018.

The aforementioned milestone was the first test balloon that showed that the idea could work. Half a year after ETHBerlin, with the support of our new partners at the Department of Decentralization, we organized a testnet conference in Berlin, the GörliCon 0, attracting more than 100 attendees. We surprised the audience by launching the Görli testnet live on stage. Lili Feyerabend pushed the red button that Ligi set up and troubleshooted. The first block was sealed on January 31, 2019, at 4:10:53 pm local time.

Soon after our big launch, Tomasz Stańczak et al. implemented Clique in the Nethermind client. Niklas Adolfsson and Hernando Castano from Parity Technologies finalized the integration with the Parity Ethereum client. The PegaSys team announced the release of a new full Ethereum client including Clique support - and with Holger Drewes surprisingly integrating Görli support in EthereumJS, Görli Testnet was able to be run in five different Ethereum clients.

Then, Görli was ready to be fully decentralized: the community quickly picked up the testnet’s governance. Nowadays, we have almost ten active validators. Faucets were created and RPC endpoints were provided.

At ETHCC 2019 in Paris, Afri Schoedon walked the audience through the success of the Görli testnet initiative. Blockscout and Etherscan provided block explorers, Infura provided API endpoints, and soon after wallets and tooling followed.

Now, a year after launch we would like to thank everyone who made this project happen. It's impossible to name all contributors and supporters. But yet we want to name a few:

María Paula Fernández and the Department of Decentralization for hosting ETHBerlin, GörliCon, and further support of the Görli initiative!
Péter Szilágyi for enduring our request to make EIP-225 an actual EIP - Thank you for writing, merging and implementing it in Geth!
André Silva and the Parity Technologies team for patiently answering all our Aura questions during the ETHBerlin 2018 hackathon!
Martin Holst Swende for helping with the Aura integration into Geth.
Elizabeth Binks, Aidan Hyman, Priom Chowdhury, David Ansermino, Dustin Brickwood, Christopher Fenos and the ChainSafe team for hacking on Görli from the start!
Jared Wasinger, Yucong Sun, Niklas Adolfsson, and Hernando Castano for implementing the most critical code in Parity Ethereum!
Lili Feyerabend for making the Görli Genesis about food and pushing the red button; and the website!
Tomasz Kajetan Stańczak and the Nethermind team, Chris McKay, Adrian Sutton, Danno Ferrin, and the PegaSys team, Holger Drewes and the EthereumJS team for your early core support!
E.G. Galano, Chris Hobcroft, Igor Barinov, Nikola Jokic, and all the other early enthusiasts and supporters!
Preston Van Loon and Prysmatic Labs for being visionary about the new testnet and planning the ETH 2.0 transition right from the start!
Mudit Gupta for gathering all resources literally on the first day!
Ronin Kaizen for running countless nodes and services since the beginning!

Thank you! This project wouldn't be possible without a community understanding the vision and carrying out the tasks together working towards a common goal.

The Görli initiative will wrap up the pending work in 2020 and is confident the testnet will be well maintained in the heart of the Ethereum ecosystem for the years to come.

If you are enthusiastic about Ethereum testnets and the Görli initiative, please attend the talk by Afri Schoedon at ETHCC 2020 in Paris who will one last time talk about testnets and why you should care.

If you feel part of the initiative, are a contributor, supporter, or evangelist, please come to Berlin in summer 2020 as DREI will feature a satellite event for celebrating the Görli testnet.

So long! 🐟

DEV Community: Afri

Getting started with Smart Infrastructure in EVE Frontier

Blockchain Sandbox: Mud & Redstone

Smart Infrastructure: Jump Gates

How to merge an Ethereum network right from the genesis block

Execution-Layer Configuration

Consensus-Layer Configuration

Launch the Network

Additional Resources

Finally authenticating Rails users with MetaMask

Web3 Concepts

Authentication Process Overview

Rails' User Model

User Controller #create

Connecting to MetaMask

User Sessions

Back and forth

What did I just read?

Security Considerations

Isn't there a standard for that?

Further resources

Finally Understanding Ethereum Accounts

Private-Public Keypairs

Addresses and Checksums

Smart-Contract Accounts

What did I just read?

Further Resources

Ethereum 2.0 Mainnet Clients

byz-f / eth2-bench-mainnet

Synchronization Metrics

Computational Resource Metrics

FAQ

q9f / eth2-bench-2020-10

How to run your own Beacon Chain

The deposit contract ceremony

The beacon chain transition specification

The genesis event

Bootstrapping the actual network

Happy Görli Testnet Launch Anniversary

User Controller `#create`