DEV Community: Sahil Sojitra

Ethereum's Consensus Puzzle: Solving the Trust Equation in a Blockchain Revolution

Sahil Sojitra — Mon, 17 Jul 2023 06:53:49 +0000

we've been talking about "consensus rules" – these are the rules that everyone must agree on for a decentralized system to work properly. Consensus is all about reaching an agreement among different participants in a distributed system so they can all have the same view of what's happening.

When it comes to decentralized record-keeping and verification, it's not enough to just trust that everything is correct. This is a challenge in decentralized networks because there's no central authority to decide what's true. That's one of the reasons why blockchain platforms are appealing – they resist censorship and don't rely on a central authority for access to information. However, this lack of a trusted authority means that any disagreements or differences have to be resolved in other ways. That's where consensus algorithms come in – they help ensure security and decentralization can work together.

Consensus is really important in blockchains because there's valuable money involved. So, in the context of blockchains, consensus is about getting everyone to agree on the same information while keeping the system decentralized. It's like having a set of strict rules without any one person or organization in charge. Instead, power and control are spread across a large network of participants who benefit by following the rules and being honest.

The ability to reach consensus across a distributed network, even when there are challenges and no central control, is the main principle behind all open public blockchains. To tackle this challenge and maintain decentralization, the blockchain community is always experimenting with different ways of achieving consensus. This chapter explores these different approaches and how they impact smart contract blockchains like Ethereum.

Consensus algorithms are an important part of how blockchains work, but they operate behind the scenes, away from the everyday use of smart contracts. It's like the plumbing of the system that you don't need to worry about. Using Ethereum is similar to using the internet - you don't need to know how the data is routed to access websites or send emails. Similarly, you don't need to understand the technical details of consensus algorithms to use Ethereum and interact with smart contracts.

Consensus via Proof Of Work (PoW)

The person who created Bitcoin also came up with a consensus algorithm called proof of work (PoW). PoW is a crucial invention that forms the foundation of Bitcoin. However, the term "mining" used for PoW can lead to a misconception about its primary purpose. People often assume that mining is mainly about creating new currency, similar to real-world mining for valuable resources. In reality, the main goal of mining (and other consensus models) is to secure the blockchain and keep control decentralized among many participants. The reward of newly minted currency serves as an incentive for those who contribute to the system's security, rather than the primary objective. The reward is a means to achieve decentralized security.

In PoW, there is also a corresponding "punishment" in the form of the energy cost required to participate in mining. If participants don't follow the rules and earn the reward, they risk losing the funds they spent on electricity for mining. Therefore, PoW consensus involves finding a balance between risk and reward, motivating participants to act honestly based on self-interest.

Currently, Ethereum operates as a PoW blockchain, using a PoW algorithm called Ethash, which has slight differences compared to Bitcoin's algorithm. Ethash serves the same purpose of securing the blockchain and decentralizing control. However, Ethereum is also exploring the possibility of transitioning to ProgPOW, a more ASIC-resistant PoW algorithm. ProgPOW is still being developed at the time of writing.

Consensus via Proof of Stake (PoS)

Before proof of work (PoW) became widely adopted, there were other consensus algorithms proposed, including variations of a consensus algorithm called proof of stake (PoS). PoW was actually created as an alternative to PoS. After Bitcoin's success, many blockchains followed its lead and used PoW. However, the research on consensus algorithms has revived interest in PoS and greatly advanced the technology.

From the beginning, Ethereum's founders intended to switch its consensus algorithm to PoS. They purposely introduced a difficulty bomb in Ethereum's PoW, which gradually makes mining more difficult, pushing the transition to PoS.

At the time of writing this book, Ethereum still relies on PoW, but the research on a PoS alternative called Casper is nearly complete. The introduction of Casper has faced delays, requiring adjustments to prevent the difficulty bomb from rendering PoW obsolete.

In general, a PoS algorithm works like this: The blockchain maintains a set of validators, and anyone holding the blockchain's main cryptocurrency (e.g., ether in Ethereum) can become a validator by locking up their cryptocurrency in a deposit. Validators take turns proposing and voting on the next valid block, and the weight of their votes depends on the size of their deposit (or stake). It's important to note that a validator risks losing their deposit if the block they staked it on is rejected by the majority of validators. On the other hand, validators receive a small reward based on their stake for every accepted block. This system of reward and punishment encourages validators to act honestly and follow the consensus rules. The main difference between PoS and PoW is that in PoS, the punishment is intrinsic to the blockchain (such as losing staked cryptocurrency), while in PoW, the punishment is extrinsic (like losing funds spent on electricity).

Ethash: Ethereum's Proof of Work Algorithm

Ethash is the proof-of-work (PoW) algorithm used in Ethereum. It combines Vitalik Buterin's Dagger algorithm and Thaddeus Dryja's Hashimoto algorithm. Ethash relies on a large dataset called the DAG, which is a directed acyclic graph. The DAG starts at around 1 GB in size and gradually grows with each epoch (30,000 blocks or approximately 125 hours).

The purpose of the DAG is to make Ethash PoW resistant to specialized mining equipment called ASICs. By depending on a large and frequently accessed data structure, Ethash aims to prevent the dominance of ASIC miners. The goal is to keep mining power decentralized and accessible to more people worldwide, using consumer-level graphics processing units (GPUs) instead. This helps avoid the concentration of mining power in the hands of a few industrial operations, as seen in Bitcoin. Ethereum’s founders wanted to avoid centralization in PoW mining, where those with access to specialized silicon fabrication factories and big budgets could dominate the mining infrastructure and undermine the security of the consensus algorithm. However, the use of GPUs for mining Ethereum caused a shortage and price increase in 2017, impacting gamers and leading to purchase restrictions.

Until recently, the threat of ASIC miners on the Ethereum network was minimal. The Ethereum development team's plans to transition to proof of stake (PoS) likely deterred ASIC suppliers. However, with other cryptocurrencies using the Ethash algorithm emerging and Ethereum Classic remaining on PoW, ASIC mining may become more prevalent on the Ethereum network before the transition to PoS. This means ASICs designed for the PoW algorithm may still find use in mining other cryptocurrencies.

Casper: Ethereum's Proof of Stake Algorithm

Casper is Ethereum's planned proof-of-stake (PoS) algorithm, designed to replace the current proof-of-work (PoW) algorithm. PoS is a different approach to achieving consensus in a blockchain network compared to PoW.

With Casper, the Ethereum blockchain will no longer rely on miners solving complex mathematical puzzles to validate transactions and create new blocks. Instead, the consensus will be achieved through a system where participants, known as validators, are chosen based on the number of cryptocurrency tokens they hold and are willing to "stake" as collateral.

Validators will lock up a certain amount of their cryptocurrency tokens in a deposit to show their commitment to the network. The weight of their influence in the consensus process will be proportional to the amount they have staked. Validators will take turns proposing and validating blocks, and the majority of validators need to agree on a proposed block for it to be accepted and added to the blockchain. Validators will be rewarded with additional cryptocurrency tokens for their honest participation, while dishonest behavior or attempts to attack the network can result in penalties, including the loss of their staked tokens.

One of the main advantages of Casper's PoS algorithm is that it requires significantly less energy consumption compared to PoW. Since there is no need for resource-intensive mining processes, the environmental impact is reduced. Additionally, Casper aims to improve the security of the network by making it more difficult and costly for attackers to compromise the consensus.

The transition to Casper is a significant milestone for Ethereum, as it will enable faster transaction confirmations, increased scalability, and a more secure and sustainable network. However, the implementation of Casper has faced delays and challenges, and the Ethereum community is actively working to address any remaining technical and economic considerations before its full deployment.

Principle of Consensus

To better understand consensus algorithms, it helps to ask a few important questions:

Who has the ability to change what happened in the past, and how? This relates to the concept of immutability in the blockchain. It's important to know who can alter or manipulate past transactions and data.
Who has the power to shape the future, and how? This refers to the concept of finality. Understanding who can determine what happens in the future and how they do it is crucial.
What is the cost involved in making changes? There may be costs associated with modifying the blockchain, such as computational power or financial expenses. Knowing the cost can provide insights into the feasibility and security of the system.
How decentralized is the power to make changes? Decentralization is an important principle in blockchain technology. It's important to assess how power is distributed among participants and whether it is concentrated in the hands of a few.
How can we detect if something has changed, and how will we know? Transparency and accountability are essential in a decentralized system. It's important to have mechanisms in place to detect and verify any changes that occur.

Consensus algorithms are continually evolving, aiming to address these questions in innovative ways. Researchers and developers are working on finding solutions that enhance immutability, increase the security of the future, minimize costs, promote decentralization, and ensure transparency in the blockchain ecosystem.

Controversy and Competition

You might be wondering why there are so many different consensus algorithms and which one is better. The truth is, there is no one-size-fits-all answer. The "best" consensus algorithm depends on various factors and involves trade-offs.

Different consensus algorithms excel in different areas, such as immutability, finality, decentralization, and cost. It's important to ask specific questions about what you want to achieve when determining which algorithm is better suited for your needs.

The design of consensus algorithms is a hotly debated topic in the blockchain industry, with billions of dollars at stake. It's an ongoing experiment where different approaches are being tested and compared. Ultimately, time will reveal which algorithms stand the test of adversarial conditions and deliver the desired results.

Conclusion

Consensus algorithms are essential in blockchain technology to ensure agreement among participants and maintain security. Different algorithms like proof of work (PoW) and proof of stake (PoS) have their strengths and weaknesses. Ethereum currently uses PoW but plans to switch to PoS with Casper. The choice of algorithm depends on factors like immutability, finality, decentralization, and cost. There is no one "best" algorithm, as it varies based on specific needs. The blockchain industry is constantly evolving, experimenting with different algorithms, and their effectiveness will be determined over time.

The Ethereum Virtual Machine: Fueling the Decentralized Revolution

Sahil Sojitra — Sun, 16 Jul 2023 12:15:29 +0000

The Ethereum Virtual Machine (EVM) is at the core of the Ethereum protocol. It acts as a powerful computation engine, somewhat similar to virtual machines used in technologies like Microsoft's .NET Framework or interpreters for programming languages like Java. In this chapter, we'll delve into the EVM, understanding its instruction set, structure, and how it operates within Ethereum's state updates.

Imagine the EVM as a virtual computer that exists within the Ethereum network. It carries out instructions and performs calculations, ensuring that transactions and contracts run as intended. This virtual machine plays a crucial role in maintaining trust and decentralization in the Ethereum ecosystem.

By grasping the fundamentals of the EVM, you'll gain insights into how Ethereum processes transactions, keeps track of its state, and enables the execution of smart contracts. Join us on an exciting exploration of decentralized computing and the technology driving Ethereum forward.

What is the EVM?

The Ethereum Virtual Machine (EVM) is responsible for handling smart contract deployment and execution within the Ethereum network. While simple transactions involving value transfers don't require the EVM, any other operation will involve the EVM computing a state update.

Think of the EVM as a giant decentralized computer on the Ethereum blockchain, consisting of numerous executable objects. Each object has its own permanent data store, making the EVM a global computational system.

The EVM operates as a quasi-Turing-complete state machine, meaning it can perform complex computations but with a limitation. The execution of smart contracts is restricted by the available gas, which determines the number of computational steps allowed. This ensures that all program executions will eventually halt, preventing any potential situation where the Ethereum platform comes to a complete halt due to endless execution.

The EVM uses a stack-based architecture, where in-memory values are stored on a stack. It employs a word size of 256 bits, primarily for efficient hashing and elliptic curve operations. The EVM comprises different components:

Immutable program code (ROM): It stores the bytecode of the smart contract being executed.
Volatile memory: This memory is initialized to zero, and its values are temporary during the execution.
Permanent storage: Part of the Ethereum state, this storage is also initialized to zero and retains values even after the execution ends.

Additionally, the EVM has a set of environment variables and data available during execution, which we will explore in more detail later in this chapter.

Comparison with Existing Technology

The term "virtual machine" is often used to describe simulating real computers or operating systems. However, the Ethereum Virtual Machine (EVM) is different. It acts as a computation engine, focused on handling calculations and storage. It's similar to the Java Virtual Machine (JVM) used for languages like Java or Scala.

The EVM executes special instructions, called bytecode, that are specific to Ethereum. Higher-level smart contract programming languages like LLL, Serpent, Mutan, or Solidity are converted into this bytecode. Just like the JVM runs Java bytecode, the EVM runs Ethereum's bytecode.

Unlike regular virtual machines, the EVM doesn't manage scheduling or interact with physical hardware. Ethereum clients handle the order of smart contracts to execute, based on verified transactions. Think of the Ethereum world computer as running one task at a time, similar to how JavaScript works. It operates virtually without needing any physical hardware to connect to.

The EVM Instruction Set (Bytecode Operations)

The EVM instruction set includes different operations that allow you to perform calculations, access information about the execution context, manipulate data in the stack, memory, and storage, control the flow of the program, and interact with other contracts.

Here are some key points about the EVM instruction set:

Arithmetic and bitwise logic operations: You can use these operations to perform basic calculations like adding, subtracting, multiplying, dividing, and finding remainders. You can also perform logical operations like AND, OR, XOR, and NOT.
Execution context inquiries: The EVM lets you ask questions about the current execution context, such as getting the address and balance of an account, finding the block number, or knowing the current gas price.
Stack, memory, and storage access: You can use instructions to manage a stack, which is a temporary storage for data. You can read from and write to memory, which is like a temporary working area. Additionally, you can read from and write to storage, which is a persistent storage associated with each contract.
Control flow operations: The EVM allows you to control the flow of the program by making decisions based on conditions, jumping to different parts of the code, and stopping execution if needed.
Logging, calling, and other operators: You can use instructions to log events, make calls to other contracts, self-destruct a contract, or handle exceptions like reverting changes.

The EVM also provides access to information about accounts, such as their addresses and balances, as well as block-related details like the block number and gas price.

The available opcodes can be divided into following categories:

Arithmetic Operations:

ADD        //Add the top two stack items
MUL        //Multiply the top two stack items
SUB        //Subtract the top two stack items
DIV        //Integer division
SDIV       //Signed integer division
MOD        //Modulo (remainder) operation
SMOD       //Signed modulo operation
ADDMOD     //Addition modulo any number
MULMOD     //Multiplication modulo any number
EXP        //Exponential operation
SIGNEXTEND //Extend the length of a two's complement signed integer
SHA3       //Compute the Keccak-256 hash of a block of memory

Stack Operations:

POP     //Remove the top item from the stack
MLOAD   //Load a word from memory
MSTORE  //Save a word to memory
MSTORE8 //Save a byte to memory
SLOAD   //Load a word from storage
SSTORE  //Save a word to storage
MSIZE   //Get the size of the active memory in bytes
PUSHx   //Place x byte item on the stack, where x can be any integer from
        // 1 to 32 (full word) inclusive
DUPx    //Duplicate the x-th stack item, where x can be any integer from
        // 1 to 16 inclusive
SWAPx   //Exchange 1st and (x+1)-th stack items, where x can be any
        // integer from 1 to 16 inclusive

Process Flow Operations:

STOP      //Halt execution
JUMP      //Set the program counter to any value
JUMPI     //Conditionally alter the program counter
PC        //Get the value of the program counter (prior to the increment
          //corresponding to this instruction)
JUMPDEST  //Mark a valid destination for jumps

System Operations:

LOGx          //Append a log record with x topics, where x is any integer
              //from 0 to 4 inclusive
CREATE        //Create a new account with associated code
CALL          //Message-call into another account, i.e. run another
              //account's code
CALLCODE      //Message-call into this account with another
              //account's code
RETURN        //Halt execution and return output data
DELEGATECALL  //Message-call into this account with an alternative
              //account's code, but persisting the current values for
              //sender and value
STATICCALL    //Static message-call into an account
REVERT        //Halt execution, reverting state changes but returning
              //data and remaining gas
INVALID       //The designated invalid instruction
SELFDESTRUCT  //Halt execution and register account for deletion

Logic Operations:

LT     //Less-than comparison
GT     //Greater-than comparison
SLT    //Signed less-than comparison
SGT    //Signed greater-than comparison
EQ     //Equality comparison
ISZERO //Simple NOT operator
AND    //Bitwise AND operation
OR     //Bitwise OR operation
XOR    //Bitwise XOR operation
NOT    //Bitwise NOT operation
BYTE   //Retrieve a single byte from a full-width 256-bit word

Environmental Operations:

GAS            //Get the amount of available gas (after the reduction for
               //this instruction)
ADDRESS        //Get the address of the currently executing account
BALANCE        //Get the account balance of any given account
ORIGIN         //Get the address of the EOA that initiated this EVM
               //execution
CALLER         //Get the address of the caller immediately responsible
               //for this execution
CALLVALUE      //Get the ether amount deposited by the caller responsible
               //for this execution
CALLDATALOAD   //Get the input data sent by the caller responsible for
               //this execution
CALLDATASIZE   //Get the size of the input data
CALLDATACOPY   //Copy the input data to memory
CODESIZE       //Get the size of code running in the current environment
CODECOPY       //Copy the code running in the current environment to
               //memory
GASPRICE       //Get the gas price specified by the originating
               //transaction
EXTCODESIZE    //Get the size of any account's code
EXTCODECOPY    //Copy any account's code to memory
RETURNDATASIZE //Get the size of the output data from the previous call
               //in the current environment
RETURNDATACOPY //Copy data output from the previous call to memory

Block Operations:

BLOCKHASH  //Get the hash of one of the 256 most recently completed
           //blocks
COINBASE   //Get the block's beneficiary address for the block reward
TIMESTAMP  //Get the block's timestamp
NUMBER     //Get the block's number
DIFFICULTY //Get the block's difficulty
GASLIMIT   //Get the block's gas limit

Ethereum State

The Ethereum Virtual Machine (EVM) is responsible for updating the Ethereum network's records as a result of executing smart contracts. This process is like a chain reaction, where transactions initiated by account holders and miners trigger changes in the overall state of the network.

At the highest level, we have the Ethereum world state, which is a mapping of Ethereum addresses to accounts. Each Ethereum address represents an account and includes information such as the account's ether balance (measured in wei), a nonce (which tracks successful transactions or created contracts), storage (used by smart contracts), and program code (for contract accounts). An externally owned account (EOA) has no code and empty storage.

When a transaction involves executing smart contract code, the EVM is set up with all the necessary information. This includes loading the code of the contract being called, initializing the program counter, loading the storage of the contract account, setting memory to zero, and setting the block and environment variables. The gas supply for the execution is also set based on the amount of gas paid by the sender. As code execution progresses, the gas supply is reduced according to the gas cost of the operations performed. If the gas supply is depleted, an "Out of Gas" exception occurs, and execution stops without applying any changes to the Ethereum state, except for incrementing the sender's nonce and deducting their ether balance. The EVM operates on a sandboxed copy of the Ethereum world state, and if execution is successful, the real-world state is updated to match the sandboxed version.

It's important to note that smart contracts can initiate further transactions, leading to a recursive process. A contract can call other contracts, and each call results in a new EVM being instantiated for the target contract. The sandbox world state of each instantiation is initialized from the sandbox of the EVM above it. Each instantiation is given a specific amount of gas, and if it runs out of gas, the sandbox state is discarded, and execution returns to the EVM above.

In simpler terms, the EVM's job is to update the Ethereum state based on the execution of smart contracts. It does this by following a set of rules and maintaining a sandboxed version of the state. Transactions trigger code execution, and if it completes successfully, the real-world state is updated accordingly. Recursive calls can occur, creating new sandboxes for each contract called, and gas limits ensure that execution doesn't continue indefinitely.

Turing Completeness and Gas

In simpler terms, being Turing complete means that a system or programming language can run any program. However, there is a problem called the halting problem, which means we can't know in advance if a program will run forever or actually finish executing.

This poses a risk for Ethereum because it operates as a single-threaded machine without a scheduler. If a program gets stuck in an infinite loop, Ethereum would become unusable.

To address this, Ethereum introduced the concept of gas. Gas provides a solution by setting a maximum amount of computation that a program can perform. If the execution of a program exceeds this limit, the Ethereum Virtual Machine (EVM) halts the execution. This means that the EVM is "quasi-Turing complete" because it can run any program as long as it terminates within a specific amount of computation.

The gas limit is not fixed in Ethereum. Users have the option to pay more to increase the limit, up to a maximum called the "block gas limit." Over time, this limit can be increased by consensus. However, at any given time, there is a limit in place, and transactions that consume too much gas during execution are stopped from running indefinitely.

In summary, gas provides a way to prevent programs from running forever and causing Ethereum to become unresponsive. It sets a maximum limit on computation, ensuring that programs have a predefined amount of resources to execute within.

Gas

In simple terms, gas is a unit of measurement in Ethereum that determines the computational and storage resources needed for actions on the Ethereum blockchain. Unlike Bitcoin, where transaction fees are based on transaction size, Ethereum considers the computational steps taken by transactions and smart contract code.

Every action in Ethereum has a fixed gas cost associated with it. Here are a few examples:

Adding two numbers costs 3 gas.
Calculating a Keccak-256 hash costs 30 gas, plus 6 gas for each 256 bits of data being hashed.
Sending a transaction costs 21,000 gas.

Gas plays two important roles in Ethereum. First, it acts as a buffer between the price of Ethereum and the rewards given to miners for their work. Second, it serves as a defense against denial-of-service attacks. To prevent infinite loops or wasteful computations, the sender of a transaction needs to set a limit on the amount of computation they are willing to pay for.

The gas system discourages attackers from sending spam transactions because they have to pay for the resources they consume. This includes computational power, bandwidth, and storage. By requiring a payment for these resources, Ethereum disincentivizes attackers from flooding the network with unnecessary or malicious transactions.

Gas Accounting During Execution

When an Ethereum Virtual Machine (EVM) executes a transaction, it is given a specific amount of gas determined by the gas limit in the transaction. Each opcode (operation) executed by the EVM has a gas cost, which reduces the available gas as the program is executed. Before each operation, the EVM checks if there is enough gas to cover its cost. If there isn't enough gas, the execution stops, and the transaction is reversed.

If the EVM successfully completes the execution without running out of gas, the gas cost used is paid to the miner as a transaction fee. The fee is calculated by multiplying the gas cost by the gas price specified in the transaction. Additionally, any remaining gas in the gas supply is refunded to the sender. The refunded gas is converted to ether based on the gas price.

miner fee = gas cost * gas price

However, if the transaction runs out of gas during execution, it is immediately terminated, triggering an "out of gas" exception. The transaction is reverted, and any changes made to the state are rolled back.

remaining gas = gas limit - gas cost
refunded ether = remaining gas * gas price

Despite the unsuccessful transaction, the sender is still charged a transaction fee because the miners have already performed computational work up to that point and need to be compensated for their efforts.

Gas Accounting Considerations

Gas costs for different operations in the Ethereum Virtual Machine (EVM) have been carefully selected to protect the Ethereum blockchain from potential attacks. These costs ensure that more computationally intensive operations require more gas.

For example, executing the SHA3 function costs 10 times more gas (30 gas) than the ADD operation (3 gas). Certain operations, like EXP, have an additional payment based on the size of the operand. Gas costs are also incurred when using EVM memory or storing data in a contract's on-chain storage.

The importance of aligning gas costs with real-world resource costs was demonstrated in 2016. An attacker exploited a discrepancy in gas costs, creating transactions that were extremely computationally expensive. This led to a severe slowdown of the Ethereum mainnet. To address this issue, a hard fork called "Tangerine Whistle" was implemented, adjusting the relative gas costs of operations.

Gas Cost Versus Gas Price

When you perform a transaction on the Ethereum network, there are two important things to consider: gas cost and gas price.

Gas cost: This is the amount of computational and storage resources needed to complete an operation on the Ethereum Virtual Machine (EVM). Different operations have different gas costs. More complex operations require more gas, while simpler operations require less.

Gas price: This is the price you are willing to pay, in ether, for each unit of gas used in your transaction. It represents the value of the computational resources you are using. You set the gas price when you send your transaction to the network.

Transaction fee: The transaction fee is the total cost of your transaction and is calculated by multiplying the gas used by the gas price. It represents the payment you make to miners for processing and confirming your transaction. Miners are more likely to include transactions with higher gas prices because it increases their potential earnings.

Gas limit: When you send a transaction, you also set a gas limit, which is the maximum amount of gas you are willing to use. It should be set higher than or equal to the expected gas usage of your transaction. If the actual gas used is less than the gas limit, you will receive a refund of the excess gas.

Although gas has a price, it cannot be possessed or used as a form of currency. Gas is simply a measurement of computational work within the Ethereum Virtual Machine (EVM). When a transaction is sent, the sender is charged a transaction fee in ether. This fee is then converted into gas for accounting purposes within the EVM. Finally, the gas fee is converted back into ether as a payment to the miners for their work in confirming and processing the transaction.

Negative Gas Costs

Ethereum rewards users for cleaning up storage and accounts by giving them refunds for the gas used during contract execution.

There are two actions in Ethereum that actually save gas costs:

When a contract is deleted (SELFDESTRUCT), the user receives a refund of 24,000 gas.
Changing a storage address from a non-zero value to zero (SSTORE[x] = 0) results in a refund of 15,000 gas.

To prevent abuse of the refund system, there is a maximum limit on the refund amount for each transaction. This limit is set to half of the total gas used in the transaction, rounded down to the nearest whole number. This ensures that refunds are fair and don't become excessive.

In simple terms, Ethereum encourages users to remove unnecessary storage and accounts by offering refunds for certain actions. Deleting a contract or setting a storage address to zero results in gas refunds. However, there is a maximum refund limit to maintain fairness and prevent misuse of the system.

Block Gas Limit

The block gas limit is the maximum amount of gas that can be used by all the transactions within a block. It determines how many transactions can fit into a block.

For example, let's say we have 5 transactions with gas limits set to 30,000, 30,000, 40,000, 50,000, and 50,000. If the block gas limit is 180,000, only four of these transactions can fit into a block, and the fifth one will have to wait for the next block. Miners decide which transactions to include in a block, and different miners may choose different combinations of transactions because they receive them in different orders from the network.

If a miner tries to include a transaction that requires more gas than the current block gas limit, the network will reject the block. Ethereum clients usually warn users with messages like "transaction exceeds block gas limit" to prevent such transactions from being issued.

On the Ethereum mainnet, the current block gas limit is around 8 million gas (according to etherscan.io at the time of writing). This means approximately 380 basic transactions, each consuming 21,000 gas, can fit into a block.

Who Decides What the block gas limit is?

Ethereum miners decide the block gas limit by voting. They use mining software to connect to Ethereum clients. The gas limit can be adjusted by a small percentage, around 0.0976%, in either direction. This allows the network's block size to be flexible based on miner decisions. The default mining strategy aims for a gas limit of at least 4.7 million gas, but it also considers the average gas usage per block using a moving average calculation over 1,024 blocks. Miners play a crucial role in determining the block gas limit and adapting the network's capacity.

Conclusions

we have delved into the Ethereum Virtual Machine (EVM) and examined how it operates when executing different smart contracts. We have observed the step-by-step execution of bytecode within the EVM. Furthermore, we have explored the concept of gas, which serves as the accounting mechanism for the EVM. Gas effectively addresses the halting problem and acts as a defense against denial-of-service attacks in the Ethereum network.

Building Trust in a Trustless World: Decentralized Applications Unveiled

Sahil Sojitra — Sun, 16 Jul 2023 04:43:45 +0000

In this chapter, we'll explore decentralized applications, or DApps. They were born from Ethereum's vision of reinventing the web and creating a new world of DApps called web3. DApps go beyond smart contracts and aim to decentralize all aspects of an application like storage, messaging, and naming.

So, what are DApps in simple terms? They are applications that don't rely on a single authority but operate in a decentralized way. Instead of one company controlling everything, DApps use technologies like smart contracts and peer-to-peer networks to give users more control over their data and interactions on the internet.

Think of it like this: with DApps, you can store your data in a decentralized manner, communicate securely without intermediaries, and have your unique identity in the network. It's all about breaking free from centralized control and giving power back to individuals.

In the next sections, we'll dig deeper into how DApps work, their reliance on smart contracts and peer-to-peer technologies, and the amazing possibilities they bring to the digital world. Get ready to discover the exciting transformation happening in the realm of decentralized applications!

What is DApp?

A DApp is an application that is mostly or completely decentralized. This means that different parts of the application, such as the backend software, frontend software, data storage, message communications, and name resolution, can be decentralized to varying degrees.

Creating a DApp offers several advantages that a typical centralized application cannot provide:

Resiliency: A DApp's backend is controlled by a smart contract and operates on a blockchain platform. This distribution ensures that the DApp remains available without any downtime as long as the platform is functioning.
Transparency: The on-chain nature of a DApp allows anyone to inspect its code, promoting trust and certainty about its functionality. Additionally, all interactions with the DApp are permanently recorded on the blockchain.
Censorship resistance: Users can interact with a DApp without interference from centralized control as long as they have access to an Ethereum node. Once a smart contract is deployed on the network, neither the service provider nor the contract owner can modify the code.

Currently, most DApps in the Ethereum ecosystem still rely on some centralized services and servers. However, in the future, we anticipate the possibility of operating every part of a DApp in a fully decentralized manner.

Backend (Smart Contract)

In a DApp, smart contracts are used to store the program code and state of the application. They replace the traditional server-side component in a regular application. However, there are some important differences to consider. Computation in smart contracts can be costly, so it's crucial to keep it minimal. It's also essential to identify which parts of the application require a trusted and decentralized execution platform.

With Ethereum smart contracts, you can create architectures where multiple smart contracts interact with each other, exchanging data and updating their own variables. The complexity is limited only by the block gas limit. Once you deploy your smart contract, other developers may use your business logic in the future.

There are two key considerations when designing smart contract architecture. First, once a smart contract is deployed, its code cannot be changed, except for complete removal if programmed with a specific opcode. It can be deleted if it is programmed with an accessible SELFDESTRUCT opcode, but other than complete removal, the code cannot be changed in any way.

Second, the size of the smart contract matters. Large monolithic contracts can be expensive to deploy and use, so some DApps choose to offload computation and use external data sources. However, relying on external data means users must trust those sources.

In summary, smart contracts are integral to DApps, but it's important to keep them efficient, plan for immutability, and carefully consider the use of external data sources for optimal DApp design.

Frontend (Web User Interface)

In a DApp, the part that users interact with can be built using normal web technologies like HTML, CSS, and JavaScript. This means that developers who are used to building websites can work on the user interface of a DApp without needing to learn new programming languages.

When it comes to communicating with Ethereum, like signing messages or making transactions, these actions can be done through a web browser using an extension called MetaMask. It helps in connecting the DApp with the Ethereum network.

Although it's possible to create DApps for mobile devices, there aren't many resources available to help developers build the user interface for mobile DApps. This is because there are limited mobile applications that can handle the specific requirements of DApps.

To connect the user interface with Ethereum, developers usually use a JavaScript library called web3.js. This library is included in the DApp's code and sent to the user's browser when they access the DApp.

In summary, building the user interface of a DApp is similar to creating a regular website using common web technologies. Interacting with Ethereum is facilitated through a browser extension like MetaMask. While mobile DApps are possible, there are fewer resources available for building their user interfaces. The web3.js library is commonly used to connect the DApp with Ethereum.

Data Storage

Smart contracts are not efficient for storing or processing large amounts of data due to high gas costs and the current low block gas limit. As a result, most DApps use off-chain data storage services. This means that instead of storing the data directly on the Ethereum blockchain, it is stored on a separate data storage platform.

There are two types of data storage platforms: centralized and decentralized. In centralized storage, data is stored on a typical cloud database, which is managed by a centralized entity. On the other hand, in decentralized storage, data is stored on a peer-to-peer (P2P) platform like IPFS (InterPlanetary File System) or Ethereum's own Swarm platform.

Decentralized P2P storage is particularly suitable for storing and distributing large static assets such as images, videos, and the resources used in the frontend of the DApp (HTML, CSS, JavaScript, etc.). These assets can be efficiently stored and accessed through decentralized storage platforms.

In the next sections, we will explore some of the options available for decentralized storage and how they can enhance the functionality and performance of DApps.

IPFS

IPFS, which stands for Inter-Planetary File System, is a decentralized storage system where files are distributed among peers in a peer-to-peer (P2P) network. It operates on a "content addressable" model, meaning that each file is given a unique hash that serves as its identifier. By using this hash, you can retrieve any file from any IPFS node in the network.

The goal of IPFS is to replace HTTP, the current protocol used for delivering web applications. Instead of storing a web application on a single server, IPFS allows the files to be stored across the network. This means that any IPFS node can serve the files, making the application more resilient and accessible.

In simpler terms, IPFS is like a giant, distributed storage system where files are identified by unique codes. You can retrieve any file by using its code, and the files are spread across many computers in a way that makes accessing them faster and more reliable. This technology aims to improve how web applications are delivered and make them more decentralized.

Swarm

Swarm is a content-addressable peer-to-peer (P2P) storage system that works similarly to IPFS. It was developed by the Ethereum Foundation as part of the Go-Ethereum suite of tools. With Swarm, you can store files that are then spread and replicated across various Swarm nodes in the network. Just like with IPFS, each file is identified by a unique hash, which allows you to access any specific file within Swarm.

The main purpose of Swarm is to enable decentralized and P2P access to websites and their associated files. Instead of relying on a central web server to host a website, Swarm allows you to access the website and its files through a decentralized network of nodes. This means that the website's content is distributed across multiple nodes, making it more resilient and ensuring that the website can be accessed even if some nodes are offline.

In simpler terms, Swarm is a technology that lets you store and access files in a decentralized way. It was created specifically to provide decentralized access to websites, so you don't have to rely on a single server. The files are spread across many nodes, ensuring better availability and reliability of the websites you visit.

Decentralized Message Communications Protocols

To communicate between different parts of an application or between users, we usually rely on a central server. However, there are decentralized alternatives that use peer-to-peer networks for messaging. One popular messaging protocol for decentralized applications is Whisper, developed by the Ethereum Foundation.

Another aspect that can be decentralized is name resolution. This means translating human-readable names, like domain names, into specific addresses on the internet. Efforts are being made to develop decentralized systems for name resolution, so we don't have to rely on a single central authority.

In simple terms, decentralized applications can communicate using peer-to-peer messaging protocols like Whisper, and there are ongoing developments to create decentralized systems for name resolution. These decentralized approaches offer alternatives to traditional centralized communication and naming systems.

The Ethereum Name Service (ENS)

When it comes to smart contracts, having a good interface for users is crucial. Just like the traditional internet uses the Domain Name System (DNS) to provide human-readable names for websites, the Ethereum blockchain has its own decentralized naming system called the Ethereum Naming System (ENS).

ENS solves the problem of translating human-readable names into specific addresses or identifiers on the Ethereum blockchain. For example, the Ethereum Foundation donation address, 0xfB6916095ca1df60bB79Ce92cE3Ea74c37c5d359, can be represented as ethereum.eth in a wallet that supports ENS.

ENS is more than just a smart contract. It's a fully functional decentralized application (DApp) that provides a decentralized name service. It is supported by various DApps that handle registration, management, and auctions of registered names. ENS demonstrates how DApps can work together, serving other DApps and being embedded within them.

In this section, we will explore how ENS works. We'll learn how to set up your own name and associate it with a wallet or Ethereum address. We'll also discover how ENS can be integrated into other DApps and how it can be used to provide easier access to resources within a DApp.

In simple terms, ENS is like a decentralized phonebook for Ethereum, allowing users to associate human-readable names with addresses on the blockchain. It's a powerful tool that enhances the usability and accessibility of DApps.

History of Ethereum Name Services

Blockchain-based name registration was initially introduced by Namecoin, which was the first non-currency application of blockchains. The Ethereum White Paper also mentioned a similar name registration system as one of its example applications.

In the early versions of Geth and the C++ Ethereum client, there was a built-in contract for name registration (although it's no longer used). Several proposals and Ethereum Request for Comments (ERC) were made regarding name services. However, it was only when Nick Johnson joined the Ethereum Foundation in 2016 that serious work on a registrar for name services began.

The Ethereum Naming System (ENS) was officially launched on May 4, 2017, which is also celebrated as Star Wars Day. Initially, there was an attempt to launch it on Pi Day, which falls on March 15, but it was unsuccessful.

The ENS Specification

ENS is defined by three Ethereum Improvement Proposals (EIPs): EIP-137 outlines the core functionalities of ENS, EIP-162 describes the auction system for the .eth root domain, and EIP-181 specifies reverse resolution of addresses.

The design philosophy behind ENS follows a "sandwich" approach. At the bottom, there is a simple layer that provides basic functionality. Above that, there are layers of more complex code that can be replaced if needed. Finally, there is a simple top layer that keeps all the funds in separate accounts.

Bottom Layer: Name Owners and Resolvers

In the Ethereum Naming System (ENS), instead of directly working with human-readable names, the system operates on "nodes." To convert a human-readable name to a node, the "Namehash" algorithm is used.

At the base layer of ENS, there is a contract defined by ERC137, which is incredibly simple, consisting of less than 50 lines of code. This contract allows only the owners of nodes to manage information about their names and create subnodes, which are similar to subdomains in the traditional Domain Name System (DNS).

The core functions in the base layer of ENS are designed to enable node owners to set information about their nodes, such as the resolver (which specifies the address of the contract that handles the node's functionality), time to live (the duration for which the information is valid), and ownership transfer. Additionally, node owners can create new subnodes and specify their owners.

The Namehash Algorithm

Namehash is a recursive algorithm that can convert any name into a hash that identifies the name.

"Recursive" means that we solve the problem by solving a subproblem that is a smaller problem of the same type, and then use the solution to the subproblem to solve the original problem.

Namehash recursively hashes the components of a name to create a fixed-length string or "node" that represents the name.Namehash recursively hashes components of the name, producing a unique, fixed-length string (or “node”) for any valid input domain. For example, the Namehash node of subdomain.example.eth is keccak('<example.eth>' node) + keccak('<subdomain>'). The subproblem we must solve is to compute the node for example.eth, which is keccak('<.eth>' node) + keccak('<example>'). To begin, we must compute the node for eth, which is keccak(<root node>) + keccak('<eth>').

The root node is what we call the "base case" of our recursion, and we obviously can’t define it recursively, or the algorithm will never terminate! The root node is defined as 0x0000000000000000000000000000000000000000000000000000000000000000(32 zero bytes).

Putting this all together, the node of subdomain.example.eth is therefore keccak(keccak(keccak(0x0...0 + keccak('eth')) + keccak('example')) + keccak('subdomain')).

Thus, mastering-ethereum.eth will be processed as follows:

namehash('mastering-ethereum.eth')
⇒ sha3(namehash('eth') + sha3('mastering-ethereum'))
⇒ sha3(sha3(namehash('') + sha3('eth')) + sha3('mastering ethereum'))
⇒ sha3(sha3(('\0' * 32) + sha3('eth')) + sha3('mastering-ethereum'))

Conclusion

Decentralized applications are the ultimate goal of Ethereum's creators since the beginning. Nowadays, many applications claim to be decentralized, but most of them are not fully decentralized. However, it is already possible to create applications that are almost completely decentralized. As technology progresses, more and more of our applications can become decentralized, resulting in a stronger, more resistant to censorship, and liberated internet.

Ethereum's Oracles: Unleashing Trustless Wisdom from Beyond the Blockchain

Sahil Sojitra — Thu, 13 Jul 2023 11:08:16 +0000

In this chapter, we'll explore oracles in Ethereum. Oracles are systems that link Ethereum smart contracts with external data sources. The term "oracle" comes from Greek mythology, where it referred to a person who communicated with gods and could see the future. In the blockchain world, an oracle serves as a connection between Ethereum and the real world by providing answers to questions beyond Ethereum.

Oracles are essential in enabling smart contracts to use real-world information, like stock prices or weather updates. This external data helps smart contracts execute actions based on certain conditions. What makes oracles fascinating is their trustless nature. They work on decentralized principles, meaning they don't rely on a single authority and don't require blind trust.

By exploring Ethereum's oracles, we aim to simplify and clarify their role in the blockchain ecosystem. Join us as we unravel the workings of these trustless systems that connect smart contracts to the outside world.

Why Oracles Are Needed

In Ethereum, there is a system called the Ethereum Virtual Machine (EVM) that runs programs and updates the Ethereum network. It follows specific rules agreed upon by the network to make sure everyone agrees on what should happen. However, there are two important limitations that come with these rules.

Firstly, the EVM cannot use a random function to generate randomness for smart contracts. If it did, different computers on the network would end up with different results, causing disagreements. This would prevent the network from reaching a shared decision on what should happen. It could quickly lead to even bigger problems, like incorrect transfers of money.

Secondly, external data, like real-time information or random numbers, can only be added to the Ethereum network through transactions. However, this data cannot be trusted because it comes from sources that cannot be verified.

To solve these issues, Ethereum uses oracles. Oracles act as a bridge between the Ethereum network and external sources of data. They provide reliable information to smart contracts, so they can make informed decisions. In the next part of this chapter, we will explore oracles in more detail and how they help overcome these limitations.

Oracles Use Cases and Examples

Oracles serve as a way to securely bring real-world information onto the Ethereum platform for smart contracts to use. They act as a bridge between off-chain data and the blockchain. By using oracles, smart contracts can enforce agreements based on real-world events and data, expanding their capabilities.

However, relying on oracles introduces certain risks to Ethereum's security. For instance, if a smart contract controls a large inheritance amount and depends on an oracle to trigger its distribution upon someone's death, there is a high incentive for hackers to manipulate the oracle and steal the assets before the intended time.

It's important to note that some oracles provide data from trusted private sources, such as universities or government departments. This data, like academic certificates or government IDs, is subjective and can only be trusted based on the authority of the source. While such data sources are not fully trustless, they are still considered oracles as they provide a bridge for smart contracts to access the data they need.

Here are some examples of the types of data that oracles can provide:

Random numbers for fair lottery selections
Triggers based on natural disasters, such as earthquake measurements for catastrophe bonds
Exchange rates for accurate cryptocurrency conversions
Capital markets data for pricing tokenized assets
Benchmark reference data, like interest rates, for smart financial derivatives
Static data like security identifiers or country codes
Time and interval data for event triggers
Weather data for insurance calculations
Information on political events for prediction markets
Sporting event outcomes for prediction markets and fantasy sports contracts
Geolocation data for supply chain tracking
Verification of damages for insurance contracts
Events happening on other blockchains for interoperability
Ether market prices for determining gas prices in fiat currency
Flight statistics for flight ticket pooling by groups or clubs.

These are just a few examples of how oracles can provide valuable information to smart contracts, enabling them to interact with the real world.

In the following sections, we will examine some of the ways oracles can be implemented, including basic oracle patterns, computation oracles, decentralized oracles.

Oracle Design Patterns

All oracles have some important functions. These include:

Collecting data from sources outside the blockchain.
Transferring the data to the blockchain using a signed message.
Making the data available by storing it in a smart contract's storage.

Once the data is stored in a smart contract, it can be accessed by other smart contracts, Ethereum nodes, or network-enabled clients. They can retrieve the data by calling a specific function in the oracle's smart contract or directly accessing the oracle's storage.

There are three main ways to set up an oracle:

Immediate-read oracles provide data that is needed for immediate decisions. For example, checking someone's age or finding the address of a website. These oracles store the data in their contract storage and allow other smart contracts or applications to look it up without a transaction. This type of oracle is useful for organizations that need to provide trusted data and eliminates the need for them to maintain their own servers. The data stored may be a hash or a summarized version to ensure efficiency and privacy.
Publish-subscribe oracles act as a broadcast service for data that changes regularly. Smart contracts can either poll the oracle on-chain or watch for updates off-chain. This pattern is similar to RSS feeds or WebSub, where the oracle updates with new information and signals that new data is available to subscribers. Interested parties can poll the oracle or listen for updates. Examples include price feeds, weather information, and social statistics. Polling is efficient in blockchain platforms like Ethereum as clients are constantly updated with state changes. Ethereum event logs make it easy for applications to receive updates from oracles.
Request-response oracles are more complex and used when the data space is too large to store in a smart contract, but users only need specific parts of the data at a time. This setup is suitable for data provider businesses. The oracle consists of on-chain smart contracts and off-chain infrastructure. When a decentralized application requests data, a transaction is initiated, specifying the requested data and additional information. The oracle processes the request, retrieves the data from an off-chain source, and returns it. The oracle may require payment, gas fees, and permissions. The resulting data is signed by the oracle owner and delivered to the decentralized application. The oracle may periodically update the data with new transactions. This setup is useful for handling large datasets and specific data requests.

For a request-response oracle, the steps can be summarized as follows:

Receive a query from a decentralized application (DApp).
Parse the query and check for payment and data access permissions.
Retrieve the relevant data from an off-chain source, encrypting it if needed.
Sign the transaction(s) with the data included.
Broadcast the transaction(s) to the network.
Schedule any additional necessary transactions, such as notifications.

Oracles can take various forms, including human, software, or hardware. They can be used to request and return data directly by an externally owned account (EOA) or through Internet of Things (IoT) devices.

The request-response pattern is commonly used in client-server architectures, allowing applications to have two-way communication. However, it may not always be suitable. For instance, if a smart bond needs an interest rate from an oracle, using a request-response pattern would require daily requests to ensure the rate is up to date. Considering that interest rates change infrequently, a publish-subscribe pattern may be more appropriate. This pattern categorizes published messages and allows subscribers to express interest in specific categories, reducing network bandwidth usage and storage costs.

In a broadcast or multicast pattern, an oracle posts messages to a channel, and subscribing contracts listen to the channel based on their subscription preferences. For example, an oracle could publish messages to a cryptocurrency exchange rate channel. A subscribing contract might request the full content of the channel for a moving average calculation, while another contract might only need the latest rate for a spot price calculation. A broadcast pattern is suitable when the oracle doesn't need to know the identity of the subscribing contract.

Data Authentication

If we assume that the data source queried by a decentralized application (DApp) is authoritative and trustworthy, there is still a concern about the trustworthiness of the oracle and the request-response mechanism operated by different entities. Data tampering during transit is a possibility, so it is crucial to have off-chain methods to ensure the integrity of the returned data. Two common approaches for data authentication are authenticity proofs and trusted execution environments (TEEs).

Authenticity proofs are cryptographic guarantees that the data has not been tampered with. They shift the trust from the data carrier to the attestor, who provides the attestation. Smart contracts can verify the authenticity proof on-chain to ensure the integrity of the data before using it. Oraclize is an example of an oracle service that utilizes various authenticity proofs. One available proof is TLSNotary, which provides evidence that HTTPS web traffic occurred between the client and a server. TLSNotary relies on TLSNotary signatures and splits the TLS master key between the server (oracle), an auditee (Oraclize), and an auditor (AWS virtual machine instance). This approach offers higher assurance against data tampering but relies on the assumption that Amazon will not tamper with the VM instance.

Town Crier is an oracle system based on the TEE approach, which uses hardware-based secure enclaves to ensure data integrity. Town Crier employs Intel's Software Guard eXtensions (SGX), which guarantees integrity and confidentiality of applications running within an enclave. SGX allows attestation, providing a digitally signed proof that an application is securely running within an enclave. Decentralized applications can verify this proof to ensure that a Town Crier instance is running securely within an SGX enclave, ensuring the authenticity of the emitted data. The confidentiality property enables Town Crier to handle private data by encrypting data queries using the instance's public key. Operating the oracle's query/response mechanism within an enclave like SGX ensures that it runs securely on trusted third-party hardware, ensuring the requested data is returned untampered (assuming trust in Intel/SGX).

In simple terms, authenticity proofs are cryptographic guarantees that data hasn't been tampered with, and trusted execution environments use secure enclaves to ensure data integrity. These approaches provide additional assurance when verifying the authenticity and integrity of data returned by oracles.

Computation Oracles

Oracles can do more than just fetch data. They can also perform complex computations that would be too expensive or impossible to do on the Ethereum blockchain. There are different approaches to achieve this.

One option is to use Oraclize, a centralized but auditable service. With Oraclize, decentralized applications can request a computation to be performed in a sandboxed AWS virtual machine. The computation is carried out within a Docker container, and the result is returned to the application. While this solution offers auditability, it is not truly decentralized.

Microsoft's ESC Framework introduces the concept of "cryptlets" as a way to provide verifiable oracle truths. Cryptlets run in an encrypted capsule and handle tasks like signing and validating messages. They support complex transactions across multiple blockchains and external systems, enabling developers to create private and portable solutions for smart contracts.

For a more decentralized approach, TrueBit offers a solution for off-chain computation. It involves solvers and verifiers who perform computations and verify them. In case of a challenge, an iterative verification process takes place on-chain. Ethereum miners act as judges to make a final ruling. TrueBit creates a computation market where decentralized applications can pay for verifiable computation outside of the network. This allows trustless smart contracts to perform various computational tasks securely.

TrueBit has applications in fields like machine learning and proof of work verification. For example, the Doge-Ethereum bridge uses TrueBit to verify Dogecoin's proof of work within a smart contract on Ethereum's Rinkeby testnet. This enables secure verification of Dogecoin transactions on the Ethereum blockchain.

In summary, oracles can perform computations, and different approaches like Oraclize, cryptlets, and TrueBit offer ways to achieve this. These methods provide options for decentralized applications to perform complex computations securely and efficiently.

Decentralized Oracles

To address the limitations of centralized oracles, decentralized approaches have been proposed. ChainLink, for example, suggests a decentralized oracle network composed of three smart contracts and an off-chain registry of data providers. The reputation contract tracks data providers' performance, while the order-matching contract selects bids from oracles based on reputation scores. The aggregation contract collects responses from multiple oracles, calculates the final result, and updates the reputation contract.

However, the challenge lies in determining how to aggregate the responses. ChainLink proposes a weighted response approach, where validity scores are assigned to oracle responses. Detecting invalid scores is difficult, as it relies on identifying outliers. Customized aggregation contracts are also allowed to address specific needs.

Another approach is the SchellingCoin protocol, where participants report values and the median value is considered correct. Reporters must provide a deposit, which is redistributed to values closer to the median. This encourages the reporting of values that align with others, aiming for a common value or Schelling point.

Jason Teutsch proposed a decentralized off-chain data availability oracle that utilizes a dedicated proof-of-work blockchain. Miners download, store, and propagate registered data to ensure local availability. While this system requires significant resources, storage can be reused by releasing data after a certain period.

In summary, decentralized oracle approaches like ChainLink, SchellingCoin, and Teutsch's proposal aim to ensure data availability and create networks of individual data providers. These methods address the limitations of centralized oracles and promote decentralized decision-making in the Ethereum network.

Conclusion

Oracles play a vital role in smart contracts by providing external information. However, they also bring risks. If you trust an oracle without caution, it can compromise the security of your smart contract by providing false data. It's important to carefully consider the trustworthiness of an oracle before using it.

Decentralized oracles can address some of these concerns by providing trustless external data to Ethereum smart contracts. By making careful choices, you can start exploring the connection between Ethereum and the real world that oracles offer.

Exploring ERC20 Tokens: The Powerhouse Behind Ethereum's Tokenized World4

Sahil Sojitra — Wed, 12 Jul 2023 13:20:53 +0000

Tokens used to refer to small, insignificant objects like transportation or arcade game tokens. But now, with blockchain technology, tokens have a new meaning. They represent assets, currency, or access rights and are not limited to specific places. Blockchain tokens can be used globally and have multiple functions. They can also be easily traded on global markets.

In this chapter, we will learn about how tokens are created and used. We will also discuss important characteristics of tokens like how they can be exchanged with each other and their inherent value. Lastly, we will explore the standards and technologies behind tokens and even try creating our own.

How Token Are Used

Tokens have many different uses, not just as a digital currency. They can do multiple things at the same time. Here are some examples:

Currency: Tokens can be used like money for buying and selling things.
Resource: Tokens can represent things you earn or produce in a sharing economy. For example, tokens could show how much storage or computer power you have to share with others.
Asset: Tokens can show that you own something valuable, like gold, real estate, or even a virtual item in a video game.
Access: Tokens can give you permission to use something, like a special website, a discussion forum, a hotel room, or a rental car.
Equity: Tokens can show that you have a share of a digital organization or a regular company.
Voting: Tokens can let you take part in making decisions in a digital system or in a legal system.
Collectible: Tokens can be special digital or physical items that people like to collect, such as unique digital artwork or rare physical objects.
Identity: Tokens can show who you are in the digital world or in the real world. It could be a picture or an official ID card.
Attestation: Tokens can be proof that something is true, like a certificate or a degree from a college.
Utility: Tokens can be used to get access to a specific service or to pay for it.

Sometimes, one token can do many of these things together. It can be hard to tell them apart because in the physical world, they are usually connected. But in the digital world, we can separate them and make them work independently. For example, we can have a token that proves something without showing who you are.

Tokens & Fungibility

Fungibility refers to the ability to interchange individual units of a token without any difference in value or function. Tokens are considered fungible when each unit can be substituted with another unit.

However, if a token's historical origin can be traced, it may not be entirely fungible. The ability to track the origin can lead to actions like blacklisting or whitelisting, which reduces or eliminates fungibility.

Non-fungible tokens (NFTs) are different because each token represents a unique item, whether tangible or intangible. For instance, a token representing ownership of a specific Van Gogh painting is not the same as a token representing a Picasso painting, even if they are part of the same "art ownership token" system. Similarly, a token representing a specific digital collectible like a CryptoKitty is not interchangeable with other CryptoKitties. Each non-fungible token has a unique identifier, such as a serial number, to differentiate it from others.

Fungibility refers to the property of a good or asset where its individual units are essentially interchangeable and indistinguishable from one another. In simple terms, it means that each unit of the item is the same as any other unit. For example, traditional currencies like dollars or euros are fungible. If you have a $10 bill, you can exchange it for another $10 bill, and they hold the same value and function.

Non-fungibility, on the other hand, refers to the uniqueness and non-interchangeability of individual units of a particular item. Each unit is distinct and cannot be substituted for another unit without differences in value or characteristics. A common example of non-fungible items are collectibles, such as trading cards or rare coins. Each trading card or coin has unique features or attributes that make it different from others, making them non-fungible.

Counterparty Risk

Counterparty risk means the risk that the other party involved in a transaction won't fulfill their obligations. Sometimes, transactions involve more than two parties, which increases this risk. For instance, if you sell a certificate of deposit for a precious metal, there are three parties involved: the seller, the buyer, and the custodian who holds the metal. The custodian's involvement adds extra risk because they have control over the physical asset.

When assets are traded using ownership tokens, there's also a risk related to the custodian. It's important to know if they actually have the asset and if they'll recognize the token transfer as a valid ownership transfer. Whether in the digital or physical world, understanding who holds the underlying asset and the rules that apply to it is crucial.

Tokens and Intrinsicality

The word "intrinsic" derives from the Latin "intra," meaning "from within."

Some tokens represent digital items that are inherent to the blockchain itself. These assets are governed by the consensus rules of the blockchain, meaning that there is no additional counterparty risk involved. For example, if you own the private keys to a CryptoKitty, you directly own the digital asset without relying on any other party. The ownership is determined by the blockchain's rules, and there is no intermediary involved.

Consensus rules are a set of agreed-upon rules in a blockchain network that determine how transactions are approved and added to the blockchain. These rules ensure that everyone in the network agrees on which transactions are valid and in what order they should be recorded.

On the other hand, many tokens are used to represent external assets like real estate, voting shares, trademarks, or gold bars. The ownership of these assets is governed by laws, customs, and policies outside the blockchain. This means that token issuers and owners may still rely on traditional legal contracts and systems. These external assets carry additional counterparty risk because they are held by custodians, recorded in external registries, or subject to laws and policies beyond the blockchain.

One significant benefit of blockchain-based tokens is the potential to convert external assets into intrinsic assets, thereby eliminating counterparty risk. For example, transitioning from equity in a traditional corporation (external) to an equity or voting token in a decentralized autonomous organization (DAO) (intrinsic) removes the need for intermediaries and reduces counterparty risk.

Using Tokens: Utility or Equity

Not all projects necessarily require tokens, and there are potential disadvantages to their use. While the idea of "tokenizing everything" may sound appealing, practical limitations exist when integrating blockchain platforms like Ethereum into existing societal structures.

Let's clarify the role of tokens in new projects. Most projects use tokens in two main ways: utility tokens and equity tokens, although these roles are often mixed up.

Utility tokens are used to gain access to specific services, applications, or resources. For instance, they might be required to use shared storage or access social media networks.

Equity tokens represent ownership or control in something, like a startup. They can range from shares that don't have voting rights but allow for dividends, to voting shares in a decentralized organization where token holders have a say in decision-making.

In summary, while tokens have utility and equity roles in projects, not all projects require tokens. The integration of blockchain platforms has limitations in their practical applicability within existing structures.

It's A Duck

Many startups face a problem: they want to use tokens to raise funds, but offering equity to the public is regulated in most places. To work around this, some startups try to call their equity tokens "utility tokens" and present them as service access vouchers. However, it's unclear if regulators will accept this strategy.

There's a saying: "If something looks and sounds like a duck, it probably is a duck." Regulators are unlikely to be fooled by these word games. They may see these attempts as trying to deceive the public by pretending the tokens are something they're not.

Tokens on Ethereum

Tokens existed before Ethereum, including the first blockchain currency, Bitcoin. Other token platforms were also built on different cryptocurrencies. However, Ethereum introduced the first token standard, which led to a huge increase in token creation.

Vitalik Buterin, one of Ethereum's founders, saw tokens as a valuable use of a programmable blockchain like Ethereum. In the early days, it was common to see Vitalik and others wearing T-shirts with the Ethereum logo and an example of a smart contract on the back, often showcasing a token implementation.

On Ethereum, tokens work differently from the native cryptocurrency called ether. The Ethereum protocol doesn't have built-in support for tokens. While sending ether is a basic function of Ethereum, tokens require the use of smart contracts. Smart contracts handle ownership, transfers, and access rights for tokens. To create a new token on Ethereum, a new smart contract needs to be developed and deployed. The smart contract defines how the token functions. It's generally recommended to follow existing token standards for compatibility and ease of use. We will discuss these standards later in the chapter and explore their benefits and drawbacks.

The ERC20 Token Standard

The first token standard on Ethereum was introduced in November 2015 by Fabian Vogelsteller as an Ethereum Request for Comments (ERC). It was given the number 20 and became known as the "ERC20 token." Most tokens today are based on this ERC20 standard.

ERC20 is a standard for fungible tokens, which means that each unit of the token is interchangeable with others and has no unique properties.

The ERC20 standard defines a common interface for token contracts, allowing compatible tokens to be accessed and used in the same way. It includes a set of required functions and events that every implementation of the standard must have, as well as optional functions and attributes that developers can add.

The required functions and events of an ERC20-compliant token contract include:

totalSupply: Returns the total number of tokens in existence.
balanceOf: Returns the token balance of a specific address.
transfer: Transfers a certain amount of tokens from the sender's address to another address.
transferFrom: Transfers tokens from one address to another, with approval.
approve: Authorizes a specific address to withdraw tokens from the sender's account.
allowance: Returns the remaining amount of tokens that a spender is allowed to withdraw from an owner's account.
Transfer: Event triggered when a transfer of tokens occurs.
Approval: Event logged when an approval is made.

In addition to these required functions, there are optional functions defined by the ERC20 standard:

name: Returns the human-readable name of the token.
symbol: Returns a symbol representing the token.
decimals: Specifies the number of decimal places used for token amounts.

These functions help ensure compatibility and consistency among ERC20 tokens. Here’s what an ERC20 interface specification looks like in Solidity:

contract ERC20 {
   function totalSupply() constant returns (uint theTotalSupply);

   function balanceOf(address _owner) constant returns (uint balance);

   function transfer(address _to, uint _value) returns (bool success);

   function transferFrom(address _from, address _to, uint _value) returns (bool success);

   function approve(address _spender, uint _value) returns (bool success);

   function allowance(address _owner, address _spender) constant returns(uint remaining);

   event Transfer(address indexed _from, address indexed _to, uint _value);

   event Approval(address indexed _owner, address indexed _spender, uint _value);
}

ERC20 Workflows: "transfer" and "approve & transferFrom"

The ERC20 token standard has two transfer functions to support different ways of using tokens.

The first way is a simple, one-step process using the transfer function. It is commonly used by wallets to send tokens from one wallet to another. This is how most token transactions happen. For example, if Alice wants to send 10 tokens to Bob, her wallet sends a transaction to the token contract, using the transfer function with Bob's address and 10 tokens as the inputs. The token contract then updates the token balances, subtracting 10 tokens from Alice's account and adding 10 tokens to Bob's account. It also records this transfer as a "Transfer" event.

The second way is a two-step process using the approve and transferFrom functions. This method allows token owners to delegate control of their tokens to another address. It is often used when a token owner wants to allow a specific contract or entity to distribute their tokens. For example, during an ICO, a company may approve a crowdsale contract to distribute tokens to buyers. The crowdsale contract can then use the transferFrom function to transfer tokens from the owner's account to each buyer's account.

These different approaches give flexibility in how tokens are used, allowing for direct transfers or delegation of control in specific scenarios like ICOs.

A Token Sale Event (TSE) is a way for companies and organizations to raise money by selling digital tokens. It is similar to when a company sells shares to investors in a stock exchange, which is called an Initial Public Offering (IPO). However, unlike IPOs that have strict rules and regulations, TSEs are more open, global, and not as organized.

To allow the AliceICO contract to sell AliceCoin tokens, Alice follows two steps. First, she creates the AliceCoin contract and keeps all the AliceCoin tokens for herself. Then, she creates the AliceICO contract, which can sell tokens for ether.

Now, Alice wants to enable the AliceICO contract to sell tokens. She sends a special request to the AliceCoin contract, telling it to allow the AliceICO contract to sell 50% of all the AliceCoin tokens. This request is recorded in an event.

When someone like Bob buys tokens from the AliceICO contract using ether, the contract needs to send AliceCoin tokens to Bob. The AliceICO contract has a set rate for converting ether to tokens. It uses this rate to calculate how many tokens Bob will receive based on the amount of ether he sends.

To make the transfer, the AliceICO contract uses a special function provided by the AliceCoin contract. It specifies Alice as the sender and Bob as the recipient. It also includes the calculated number of tokens to transfer. This transfer is recorded in an event.

The AliceICO contract can repeat this process multiple times as long as it doesn't sell more tokens than what Alice approved. It can keep track of the available tokens by checking the approved amount.

In simple terms, Alice allows the AliceICO contract to sell her AliceCoin tokens. When someone buys tokens, the AliceICO contract calculates how many tokens to give based on the amount of ether. The tokens are transferred from Alice to the buyer using special functions, and the contract can sell tokens as long as it doesn't go over the approved limit.

Issues with ERC20 Tokens

ERC20 tokens have become very popular, with many tokens being created for various purposes like crowdfunding and auctions. However, there are some issues and differences compared to regular ether transactions.

When you transfer ether, you send it directly to someone's address. But with ERC20 tokens, the transfers happen within the token contract itself, not directly to the recipient's address. The token contract keeps track of balances and events. So, when you transfer tokens, no transaction is actually sent to the recipient. Instead, the recipient's address is added to a list within the token contract. This means that the token contract changes, but not the recipient's address. Some wallets may not show your token balance unless you add the specific token contract to the wallet.

There are differences between tokens and ether. Ether can be sent and received easily, but tokens need special functions like "transfer" or "approve & transferFrom" that exist only in the ERC20 contract. These token transfers don't trigger any actions in the recipient's contract. It can be confusing because tokens are meant to function like cryptocurrencies, but they have these differences.

Another issue is that even when you want to send tokens, you still need to have some ether to pay for transaction fees. Tokens cannot be used to directly pay for fees, and the token contract can't pay for fees on your behalf. This can create confusion, especially if you didn't realize you needed ether to send tokens.

These issues are specific to ERC20 tokens and can be challenging to handle. They may require changes to the token interface or even to the Ethereum network itself. Some issues may not have a perfect solution, so user interfaces may need to be designed to make things easier and provide a consistent experience for users.

ERC223: A Proposed Token Contract Interface Standard

The ERC223 proposal addresses the issue of accidentally sending tokens to contracts that may not support them. It does this by checking whether the destination address is a contract or not. In ERC223, contracts that want to receive tokens must have a function called "tokenFallback" implemented. If the recipient of a token transfer is a contract and it doesn't have support for tokens (meaning it doesn't have the tokenFallback function), the transfer will fail. This helps prevent tokens from being lost or stuck in contracts that cannot handle them properly.

The ERC223 contract interface specification is:

interface ERC223Token {
  uint public totalSupply;

  function balanceOf(address who) public view returns (uint);

  function name() public view returns (string _name);

  function symbol() public view returns (string _symbol);

  function decimals() public view returns (uint8 _decimals);

  function totalSupply() public view returns (uint256 _supply);

  function transfer(address to, uint value) public returns (bool ok);

  function transfer(address to, uint value, bytes data) public returns (bool ok);

  function transfer(address to, uint value, bytes data, string custom_fallback)public returns (bool ok);

  event Transfer(address indexed from, address indexed to, uint value,bytes indexed data);
}

ERC223 is not widely implemented, and there is some debate in the ERC discussion thread about backward compatibility and trade-offs between implementing changes at the contract interface level versus the user interface.

ERC777: A Proposed Token Contract Interface Standard

ERC777 is a proposal to make token contracts better. It has a few goals:

It works with existing ERC20 tokens, so they can still be used.
Tokens can be sent using a "send" function, just like how we send ether.
It can be registered with ERC820, which helps keep track of token contracts.
Contracts and addresses can decide which tokens to send using a special function called "tokensToSend".
Contracts and addresses can know when they receive tokens by having a function called "tokensReceived". This prevents tokens from getting stuck in contracts.
Existing contracts can use proxy contracts for the special token functions.
It doesn't matter if tokens are sent to a contract or a regular address, it works the same way.
There are specific events for creating and destroying tokens, so we can keep track of what happens to them.
Trusted third parties called operators can move tokens for token holders.
Extra information can be added to token transactions using special fields called "userData" and "operatorData".

In simple terms, ERC777 aims to improve token contracts by making them work better with existing tokens, allowing easier token transfers, giving more control over tokens, and adding useful features like events and extra transaction information.

The ERC777 contract interface specification is:

interface ERC777Token {
    function name() public constant returns (string);

    function symbol() public constant returns (string);

    function totalSupply() public constant returns (uint256);

    function granularity() public constant returns (uint256);

    function balanceOf(address owner) public constant returns (uint256);

    function send(address to, uint256 amount, bytes userData) public;

    function authorizeOperator(address operator) public;

    function revokeOperator(address operator) public;

    function isOperatorFor(address operator, address tokenHolder) public constant returns (bool);

    function operatorSend(address from, address to, uint256 amount, bytes userData,bytes operatorData) public;

    event Sent(address indexed operator, address indexed from, address indexed to, uint256 amount, bytes userData, bytes operatorData);

    event Minted(address indexed operator, address indexed to, uint256 amount, bytes operatorData);

    event Burned(address indexed operator, address indexed from, uint256 amount, bytes userData, bytes operatorData);

    event AuthorizedOperator(address indexed operator, address indexed tokenHolder);

    event RevokedOperator(address indexed operator, address indexed tokenHolder);
}

Using Token Standards

In the previous section, we discussed various proposed and widely used standards for token contracts. Now, let's explore what these standards do, whether you should use them, how to use them, and if you should add extra functionality beyond these standards. We will also consider which standards to use.

What Are Token Standards? What Is Their Purpose?

Token standards provide a set of rules that token contracts must follow. For example, the ERC20 standard specifies the functions and behavior a token contract should have. You can also add extra features if you want.

The main purpose of these standards is to make different contracts work together smoothly. Wallets, exchanges, and other tools can easily interact with contracts that follow the standard. If you create a token contract that follows the ERC20 standard, people can use it with existing wallets without any extra work.

These standards describe what contracts should do, but not how they should be made. You have the freedom to decide how to implement the functions inside your contract. The standards provide some requirements for how the contract should behave in certain situations, but they don't give specific instructions. For example, they might specify how the transfer function should handle a zero value.

In simple terms, token standards are like a set of rules that token contracts should follow. They help different contracts work well together. You can add more features if you want, and the standards describe what the contract should do, but not how to make it.

Should You Use These Standards?

Every developer faces a tough decision: should they stick to existing standards or go beyond them to create something new and innovative?

This decision is not easy to make. Standards by their nature limit your ability to innovate because they provide a specific framework that you must follow. However, these standards have been developed based on the experience of many applications and usually work well for most use cases.

There is a bigger issue to consider as well: the importance of interoperability and widespread adoption. If you choose to use an existing standard, you benefit from the existing systems and tools designed to work with that standard. If you deviate from the standard, you have to consider the cost of building your own support infrastructure or convincing others to support your new standard. Choosing to ignore existing standards and creating everything from scratch is known as "Not Invented Here" syndrome, and it goes against the principles of open source culture. However, progress and innovation sometimes require breaking away from tradition. It's a challenging choice that requires careful consideration.

Security by Maturity

When implementing a standard like ERC20, you have two options: use an existing implementation or create your own. This choice has important security implications.

Existing implementations have been thoroughly tested and used in real-world scenarios. They have proven to be secure and reliable, handling millions of dollars' worth of tokens. Creating your own implementation is challenging and risky, as there are many ways it can be compromised. It is safer to use a well-established implementation like OpenZeppelin's ERC20, which focuses on security.

If you choose an existing implementation, you can still add your own custom features if needed. However, be cautious, as adding complexity increases the chance of introducing vulnerabilities. Each line of code you add expands the potential risks. You might not discover problems until your contract holds significant value and someone exploits it.

Security by Maturity

When implementing a standard like ERC20, you have two options: use an existing implementation or create your own. This choice has important security implications.

Extensions to Token Interface Standards

The token standards we discussed provide a basic set of functions, but many projects have created their own versions with more features. These additional features include:

Owner control: The ability to give specific addresses special powers like blacklisting or minting tokens.
Burning: The ability to intentionally destroy tokens by sending them to an unspendable address or reducing the total supply.
Minting: The ability to create new tokens and increase the total supply.
Crowdfunding: The ability to sell tokens through auctions or other methods.
Caps: Setting limits on the maximum token supply.
Recovery backdoors: Allowing designated addresses to recover funds or reverse transactions if needed.
Whitelisting: Restricting token transfers to specific addresses, often used for vetted investors.
Blacklisting: Preventing token transfers to specific addresses.

There are example implementations available, but no widely accepted standard for these features.

When deciding whether to add extra features to a token standard, developers need to consider the trade-off between innovation and risk, as well as interoperability and security. It's important to carefully weigh the benefits and potential drawbacks before implementing additional functionality.

Tokens and ICOs

Tokens have become an important part of Ethereum, but not all tokens are trustworthy. Many tokens being offered now are scams or schemes to make quick money.

It's important to understand that the long-term impact and potential of tokens and the technology behind them is different from the current situation. Some tokens are fraudulent or designed to trick people.

We should recognize the potential of tokens and the platform they are built on, but also be aware of the scams and fraudulent activities happening now. Token standards and the platform have the potential to bring big changes in the future, but we need to be cautious and careful in the present.

Conclusion

Tokens are a really useful thing in Ethereum. They can be the starting point for many important applications that work without any central control. In this chapter, we learned about different kinds of tokens and standards. You even got to make your own token and build something with it.

From Bits to Blockchain: The Inner Workings of Ethereum Transactions Explained

Sahil Sojitra — Tue, 11 Jul 2023 10:51:12 +0000

Transactions in Ethereum are like messages that cause changes on the blockchain. They are created by user accounts and processed by the Ethereum network. Transactions are what make things happen and execute contracts on Ethereum.

Imagine Ethereum as a machine with different states. Transactions are what drive this machine and make it change from one state to another. Contracts, which are important parts of Ethereum, cannot work by themselves. Ethereum doesn't run automatically; everything starts with a transaction.

In this blog, we will explain transactions, how they work, and their details. While some of this information is more relevant to those building wallet apps or dealing with transactions at a technical level, you don't need to worry about it if you use existing wallet applications. However, you might still find the details interesting!

The Structure of a Transaction

A transaction in Ethereum has a specific structure when it's sent over the network. However, different clients and applications may add their own extra information when storing and processing the transaction internally.

Here are the main components of a transaction:

Nonce: A sequence number assigned by the sender's account to prevent duplicate transactions.
Gas price: The amount of ether (in wei) that the sender is willing to pay for each unit of gas used to process the transaction.
Gas limit: The maximum amount of gas the sender is willing to use for this transaction.
Recipient: The Ethereum address of the intended receiver of the transaction.
Value: The amount of ether (in wei) to be sent to the recipient.
Data: A variable-length payload containing binary information.
v, r, s: Three components used for the digital signature of the sender's account.

The transaction's structure is serialized using a special encoding scheme called Recursive Length Prefix (RLP). Ethereum uses big-endian integers to represent numbers, and the length of each field is identified using RLP's length prefix.

It's important to note that the field labels (e.g., to, gas limit) mentioned here are for clarification purposes and are not part of the serialized transaction data itself. RLP doesn't include field delimiters or labels. Anything beyond the specified length belongs to the next field in the structure.

While this is the structure of the transmitted transaction, most software representations and user interfaces add extra information derived from the transaction or the blockchain.

For example, you might notice that the sender's address ("from" data) is not explicitly included in the transaction. That's because the sender's public key can be derived from the ECDSA signature components (v, r, s), and the address can be derived from the public key. The "from" field you see in transaction visualizations is added by the software for clarity. Other additional information, such as the block number and transaction ID, is often added by client software but is not part of the original transaction message itself.

The Transaction Nonce

The nonce is an important component of a transaction, but it is often misunderstood. It represents the number of transactions sent from an address or the number of contract creations made by that address. However, the nonce itself is not stored explicitly on the blockchain. It is dynamically calculated by counting the confirmed transactions originating from an address.

The nonce serves two purposes: maintaining the order of transactions and preventing duplication. Let's consider examples for each scenario:

Transaction Order: Imagine you want to send two transactions: one for 6 ether and another for 8 ether. You send the 6-ether transaction first, thinking it's more important, and then the 8-ether transaction. However, if your account doesn't have enough funds for both, one transaction will fail. Since transactions can reach nodes in different orders, it's uncertain which one will be accepted. But with the nonce, the first transaction will have a specific nonce value (let's say 3), while the second transaction will have the next nonce value (4). So, the second transaction will be ignored until the previous nonces are processed, ensuring the desired order of execution.
Duplication Protection: Suppose you send a payment of 2 ether to someone for a product. Without a nonce, a second transaction with the same amount to the same address would appear identical to the first one. This means anyone on the network could replay your transaction multiple times, potentially draining your funds. However, with the nonce value included in the transaction data, every single transaction is unique, even when sending the same amount of ether to the same recipient address multiple times. Even if you send the same amount to the same recipient multiple times, the incrementing nonce ensures no duplication. This safeguards your payments from being exploited or replayed by others.

To summarize, the nonce is crucial in account-based protocols like Ethereum, unlike the "Unspent Transaction Output" (UTXO) mechanism used in Bitcoin. It maintains transaction order and prevents unauthorized duplication, enhancing the security and integrity of transactions.

Keeping Track of Nonces

The nonce represents the count of confirmed transactions originating from an account. You can obtain the nonce by querying the blockchain through a web3 interface.

>web3.eth.getTransactionCount("0x9e713963a92c02317a681b9bb3065a8249de124f")
49

The nonce is a zero-based counter, meaning the first transaction has nonce 0. In this example, we have a transaction count of 49, meaning nonces 0 through 48 have been seen. The next transaction’s nonce will need to be 49.

Your wallet software keeps track of nonces for each managed address. If you're building your own wallet or application, you assign the next nonce when creating a new transaction. However, the pending transactions may not be included in the nonce count immediately. It's important to wait until the pending and confirmed counts are equal before relying on the nonce from the getTransactionCount call. Once the counts align, you can start tracking the nonce in your application until each transaction is confirmed.

>web3.eth.getTransactionCount("0x9e713963a92c02317a681b9bb3065a8249de124f", \
"pending")
40
>web3.eth.sendTransaction({from: web3.eth.accounts[0], to: \
"0xB0920c523d582040f2BCB1bD7FB1c7C1ECEbdB34", value: web3.utils.toWei(0.01, "ether")});
>web3.eth.getTransactionCount("0x9e713963a92c02317a681b9bb3065a8249de124f", \
"pending")
41
>web3.eth.sendTransaction({from: web3.eth.accounts[0], to: \
"0xB0920c523d582040f2BCB1bD7FB1c7C1ECEbdB34", value: web3.utils.toWei(0.01, "ether")});
>web3.eth.getTransactionCount("0x9e713963a92c02317a681b9bb3065a8249de124f", \
"pending")
41
>web3.eth.sendTransaction({from: web3.eth.accounts[0], to: \
"0xB0920c523d582040f2BCB1bD7FB1c7C1ECEbdB34", value: web3.utils.toWei(0.01, "ether")});
>web3.eth.getTransactionCount("0x9e713963a92c02317a681b9bb3065a8249de124f", \
"pending")
41

When we sent the first transaction, the transaction count increased to 41, indicating a pending transaction. However, when we quickly sent three more transactions, the getTransactionCount call didn't include them. It only counted one transaction, even though we expected all three to be pending in the network's memory pool (mempool). If we wait for a few seconds to let network communications settle, the getTransactionCount call will return the expected number. But during this time, when there are multiple pending transactions, relying on getTransactionCount may not be helpful.

When developing an application that creates transactions, it's important not to rely on getTransactionCount for pending transactions. Only when the pending and confirmed counts are the same (all outstanding transactions are confirmed) can you trust the getTransactionCount result and start tracking the nonce. Afterward, you should keep track of the nonce in your application until each transaction is confirmed.

Gaps in Nonces, Duplicate Nonces and Confirmation

It's crucial to keep track of nonces when creating transactions programmatically, especially if you're doing so from multiple independent processes simultaneously.

The Ethereum network processes transactions based on their nonce in a sequential manner. If you send a transaction with nonce 0 and then another with nonce 2, the second transaction will not be included in any blocks. It will be stored in the mempool, and the network will wait for the missing nonce (1 in this case) to appear. Nodes assume that the missing nonce is delayed and that the transaction with nonce 2 arrived out of order.

To resolve this, you need to send a valid transaction with the missing nonce (1) to allow both transactions (1 and 2) to be processed and included in blocks. If there's a gap in the nonce sequence, subsequent transactions will be stuck, waiting for the missing nonce to be filled. It's important to note that once a transaction with the missing nonce is validated, all the subsequent transactions with higher nonces become valid. It's not possible to undo or recall a transaction.

On the other hand, if you accidentally duplicate a nonce by sending two transactions with the same nonce but different recipients or values, one of them will be confirmed, and the other will be rejected. The confirmation of the transaction will be determined by the order in which they reach the first validating node, making it somewhat random.

As you can see, accurately managing nonces is necessary to avoid issues. If your application doesn't handle nonces correctly, you may encounter problems. Handling nonces becomes even more challenging when dealing with concurrent transactions, as we'll explore in the next section.

Concurrency, Transaction Origination, and Nonces

Concurrency means multiple independent systems working at the same time. In Ethereum, different parts of the system can work simultaneously, but they all follow the same rules to maintain consistency.

Imagine you have multiple wallet applications generating transactions from the same address. For example, an exchange processing withdrawals from its online wallet. Ideally, you'd want multiple computers handling withdrawals to avoid slowdowns or failures. However, this creates problems because each computer needs to coordinate and assign nonces correctly.

One approach is to use a single computer to assign nonces to the computers generating transactions. However, this creates a single point of failure. If a nonce is assigned but not used, all subsequent transactions get stuck.

Another approach is to generate transactions without assigning a nonce, leaving them unsigned. These unsigned transactions can then be queued to a single node that signs them and keeps track of nonces. However, this creates a bottleneck as signing and tracking nonces can become congested under heavy load. This approach lacks concurrency in a critical part of the process.

These concurrency challenges, combined with the difficulty of tracking balances and confirmations in independent processes, often lead to solutions that avoid concurrency. For example, some implementations use a single process to handle all withdrawals or set up multiple independent wallets that require occasional rebalancing.

In summary, managing concurrency in Ethereum can be complex. Practical solutions aim to balance between concurrent processing and avoiding bottlenecks in order to maintain a stable and reliable system.

Transaction Gas

Gas is like the fuel used in Ethereum transactions. It's not the same as ether, the main cryptocurrency of Ethereum, but a separate virtual currency with its own value compared to ether. Gas is used to control the resources that a transaction can use because it's processed by many computers worldwide. This ensures that transactions don't overload the system or consume excessive resources.

Gas exists separately from ether to protect the system from the rapid changes in ether's value. It also helps manage the costs of different resources involved in transactions, such as computation, memory, and storage.

The gasPrice field in a transaction allows the sender to choose the price they are willing to pay for gas. This price is measured in wei per unit of gas. For example, if a transaction sets the gasPrice to 3 gwei (3 billion wei), it means the sender is willing to pay that amount in exchange for the required gas.

Wallets can adjust the gasPrice in transactions to influence the speed of confirmation. Higher gasPrice means faster confirmation, while lower prices result in slower confirmation. In some cases, transactions with zero gasPrice can be fee-free and still get included in blocks during periods of low demand.

The gasPrice can be set to zero, which means wallets can create transactions without paying any fees. However, these transactions may not get confirmed if the network is busy. The Ethereum protocol doesn't prevent free transactions, and some examples of such transactions have been successfully added to the Ethereum blockchain.

The gasLimit field determines the maximum amount of gas the sender is willing to buy for the transaction. For simple payments, the gas amount needed is fixed at 21,000 units. To calculate the cost in ether, multiply 21,000 by the gasPrice. For example:

> web3.eth.getGasPrice(function(err, res){console.log(res*21000)})
> 210000000000000

When interacting with a contract, estimating the gas needed becomes challenging. Contracts can have different execution paths and varying gas costs based on conditions beyond your control. For instance, a contract may have different outcomes after a certain number of calls, resulting in different gas costs depending on prior transactions. The gas needed for a contract cannot be accurately determined.

An analogy often used is to think of gasLimit as the fuel tank capacity for your car (representing the transaction). You fill the tank with an estimated amount of gas needed for the journey (computation for validating the transaction). However, unexpected factors like diversions or complex execution paths can increase fuel consumption.

In Ethereum, the gasLimit works more like a credit account at a gas station. When you send a transaction, it's validated that your account has enough ether to cover gasPrice * gasLimit. The actual deduction from your account occurs after the transaction finishes executing, and you're billed only for the gas consumed. Therefore, you need to have enough balance to cover the maximum amount you're willing to pay before sending the transaction.

Transaction Recipient

The recipient of a transaction is specified in the "to" field, which contains a 20-byte Ethereum address. This address can belong to an individual (EOA) or a contract.

Ethereum doesn't perform any validation on this field. Any 20-byte value is considered a valid address. Whether the address corresponds to an existing private key or a contract is not verified by Ethereum. If you send a transaction to the wrong address, the ether sent will likely be lost forever and cannot be accessed, as most addresses do not have a known private key to authorize spending.

Validating the address is typically done at the user interface level to prevent such mistakes. Burning ether, or making it unspendable, can have valid reasons in some cases, like discouraging cheating in payment channels or other smart contracts. Since the total amount of ether is finite, burning it effectively redistributes its value to all ether holders in proportion to the amount they hold.

Transaction Value and Data

The main "payload" of a transaction is contained in two fields: value and data. Transactions can have both value and data, only value, only data, or neither value nor data. All four combinations are valid.

A transaction with only value is a payment. A transaction with only data is an invocation. A transaction with both value and data is both a payment and an invocation. A transaction with neither value nor data—well that’s probably just a waste of gas! But it is still possible.

Transmitting Value to EOA's and Contracts

When you send a transaction with a value in Ethereum, it can be considered a payment. The behavior of such transactions depends on whether the destination address is a contract or an Externally Owned Account (EOA).

For EOA addresses, the value you send is added to the recipient's balance. If the address is new and hasn't been seen before, it will be added to the client's internal representation of the state, and its balance will be initialized with the payment value.

If the destination address is a contract, the Ethereum Virtual Machine (EVM) will execute the contract and attempt to call the function specified in the transaction's data payload. If no data is provided, the EVM will try to execute a fallback function. If the fallback function allows payments, it will be executed to determine the next steps. If there is no code in the fallback function or if it is not payable, the transaction will be reverted.

A contract can reject incoming payments by throwing an exception when a function is called or based on specific conditions coded in a function. If the function executes successfully without exceptions, the contract's state will be updated to reflect an increase in its ether balance.

In simpler terms, when you send ether to an individual, it increases their balance. But if you send ether to a contract, the contract's code is executed, and if the contract allows payments, its balance will increase. Contracts can also reject payments based on their code's conditions.

Transmitting a Data Payload to an EOA or Contract

When you include data in your transaction, it is usually intended for a contract address. Sending data to an externally owned account (EOA) is also valid, but its interpretation depends on the wallet you use. The Ethereum protocol ignores the data when sent to an EOA, and most wallets also ignore any data received by EOAs they control. In the future, there might be standards that allow wallets to interpret data like contracts do, enabling transactions to invoke functions within user wallets. However, it's important to note that the interpretation of data by an EOA is not governed by Ethereum's consensus rules, unlike contract executions.

Assuming your transaction is targeting a contract address, the Ethereum Virtual Machine (EVM) interprets the data as a contract invocation. In most cases, the data represents a function call, specifying the function's name and any encoded arguments.

The data payload sent to an ABI-compatible contract (which most contracts are) is a hexadecimal serialization that consists of two parts:

Function Selector: The first 4 bytes of the Keccak-256 hash of the function's prototype. This uniquely identifies the function you want to invoke.
Function Arguments: The arguments of the function, encoded based on the rules defined in the ABI specification for various data types.

In simpler terms, when you send data to a contract, it is treated as a request to call a specific function within that contract, and the data payload contains the necessary information to identify the function and pass its arguments.

Special Transaction: Contract Creation

When you want to create a new contract on the Ethereum blockchain, you send a special transaction to an address known as the zero address, represented by 0x0 in the "to" field. This address is not associated with an externally owned account (EOA) or an existing contract. It serves only as a destination for contract creation.

While the zero address is specifically meant for contract creation, it sometimes receives accidental payments from various addresses. This can result in the loss of ether if sent by mistake. On the other hand, intentional ether burns, where ether is deliberately destroyed by sending it to an address that cannot spend it, can also occur. However, if you intend to perform an intentional ether burn, it is recommended to use a designated burn address to make your intention clear to the network.

0x000000000000000000000000000000000000dEaD

Any ether sent to the designated burn address will become unspendable and be lost forever.

To create a contract, your transaction only needs to include a data payload containing the compiled bytecode that will generate the contract. The purpose of this transaction is solely to create the contract. Optionally, you can specify an ether amount in the value field to set the new contract with an initial balance. However, sending ether to the contract creation address without a data payload (no contract) has the same effect as sending it to a burn address—it will be lost as there is no contract to credit.

Digital Signatures

In this section, we look at how digital signatures work and how they can be used to present proof of ownership of a private key without revealing that private key.

The Elliptic Curve Digital Signature Algorithm (ECDSA)

The digital signature algorithm used in Ethereum is called the Elliptic Curve Digital Signature Algorithm (ECDSA). It relies on private-public key pairs based on elliptic curves. A digital signature in Ethereum serves three purposes: proving authorization, ensuring non-repudiation, and guaranteeing data integrity.

A digital signature is a mathematical scheme that verifies the authenticity of digital messages or documents. It consists of two parts: creating the signature using a private key and verifying the signature using a public key.

Creating a Digital Signature

In Ethereum, when creating a digital signature, the transaction is used as the message, specifically the Keccak-256 hash of the RLP-encoded transaction data. The private key of the account is used for signing, resulting in the signature. The signature is composed of two values, often referred to as "r" and "s."

Sig = F-sig(F-keccak256(m),k)

where,

k is the signing private key.
m is the RLP-encoded transaction.
F-keccak256 is the Keccak-256 hash function.
F-sig is the signing algorithm.
Sig is the resulting signature.

The function F-sig produces a signature Sig that is composed of two values, commonly referred to as r and s:

Sig = ( r , s )

Verifying the Signature

To verify a signature, you need the signature itself (represented by "r" and "s"), the serialized transaction, and the public key that corresponds to the private key used to create the signature. Verification ensures that only the owner of the private key that generated the public key could have produced the signature for that specific transaction.

The signature verification algorithm takes the message (which is a hash of the transaction), the public key of the signer, and the signature (r and s values). If the algorithm determines that the signature is valid for the given message and public key, it returns "true."

ECDSA Math

The signature algorithm involves generating a ephemeral (temporary) private key to ensure the security of the sender's actual private key. This temporary key is used to calculate the values "r" and "s" in the signature.

Here's a simplified explanation of the steps involved:

Generate a random number "q" as the temporary private key.
Calculate the corresponding temporary public key "Q" using the elliptic curve generator point.
The "r" value of the signature is the x coordinate of the temporary public key "Q".
Calculate the "s" value of the signature using the formula:

s ≡ q-1 (Keccak256(m) + r * k) (mod p).

where,

"q" is the temporary private key.
"r" is the x coordinate of the temporary public key.
"k" is the actual private key of the sender (EOA owner).
"m" is the transaction data.
"p" is the prime order of the elliptic curve.

To verify a signature, the process is the inverse of the signature generation function. It involves using the "r" and "s" values of the signature, along with the sender's public key, to calculate a point on the elliptic curve called "Q," which represents the ephemeral public key used during signature creation. Here are the steps:

Check that all inputs are properly formatted.
Calculate the value "w" by taking the inverse of "s" modulo "p".
Calculate "u1" as the result of multiplying the Keccak256 hash of the signed transaction data ("m") by "w" modulo "p".
Calculate "u2" as the result of multiplying "r" by "w" modulo "p".
Finally, calculate the point "Q" on the elliptic curve using the formula

Q ≡ u1 * G + u2 * K (mod p).

where,

"r" and "s" are the signature values.
"K" is the sender's public key (EOA owner's public key).
"m" is the signed transaction data.
"G" is the elliptic curve generator point.
"p" is the prime order of the elliptic curve.

If the x coordinate of the calculated point "Q" matches the "r" value of the signature, then the verifier can conclude that the signature is valid.

Separating Signing and Transmission (Offline Signing)

Once a transaction is signed, it can be transmitted to the Ethereum network. Normally, the process of creating, signing, and broadcasting a transaction happens in a single step. However, it is possible to separate the signing and transmission steps for security reasons.

Separating these functions is done to protect the private keys used for signing. The computer that signs the transaction needs to have the private keys loaded in memory, which can be risky if it is connected to the internet. By performing the signing on an offline device and the transmission on an online device, known as offline signing, the private keys remain secure.

Here's an overview of the offline signing process:

Create an unsigned transaction on an online computer where the current account state, including the nonce and available funds, can be retrieved.
Transfer the unsigned transaction to an offline device that is not connected to the internet for signing. This can be done through methods like using a QR code or a USB flash drive.
Transmit the signed transaction back to an online device for broadcasting it on the Ethereum blockchain. This can be done by scanning a QR code or transferring the transaction via a USB flash drive.

By following this process, the private keys are kept offline during the signing step, reducing the risk of unauthorized access to the keys.

Transaction Propagation

The Ethereum network uses a system where all computers in the network, called nodes, are connected to each other. When someone creates or sends a transaction, it spreads across the network like a wave, reaching every node. Each node checks if the transaction is valid and then passes it on to its connected nodes. This process continues until every node has a copy of the transaction.

The way transactions spread is designed to be fast and efficient. It ensures that within a few seconds, the transaction is known to every node in the network around the world. The best part is that no node can tell where the transaction originated from. This makes it difficult for anyone to track or interfere with transactions unless they control a large portion of the network.

This approach helps maintain security and privacy in the Ethereum network. It ensures that transactions are widely distributed and protected from unauthorized access or manipulation.

Recording on the Blockchain

In the Ethereum network, some nodes are operated by miners who use powerful computers to process transactions and add them to blocks. These computers, known as mining farms, work to find a proof of work that validates the transactions in a block. Valid transactions are then recorded in the Ethereum blockchain.

Once a transaction is included in a block and mined, it becomes a permanent part of the blockchain. It can modify the state of the Ethereum network by changing account balances or invoking contracts that make internal changes. These changes are recorded in a transaction receipt, which also includes any associated events.

In simpler terms, when a transaction completes its journey from creation to being mined, it leaves a lasting impact on the Ethereum network and becomes a permanent part of the blockchain.

Multiple-Signature (Multisig) Transaction

In Ethereum, it is possible to create smart contracts that enforce custom rules for transferring ether and tokens. This allows for the implementation of features like multisignature accounts, where funds can only be spent when multiple parties sign a transaction.

To set up a multisignature account, you transfer your money to a special contract instead of a regular account. This contract is programmed with the rules you want, such as requiring two or more people to sign off on transactions. When you want to send money from the multisignature account, all the authorized users need to sign and approve the transaction using a wallet app.

These contracts can also be designed to require multiple signatures for executing specific actions or triggering other contracts. The security of the multisignature system depends on the code written for the contract.

While this flexibility is powerful, it also introduces the risk of programming errors that could undermine the security of multisignature setups. There are proposals to create a built-in feature for multisignature transactions in Ethereum to simplify the process and make it more secure, similar to Bitcoin's system that is known for its robustness and security.

Conclusion

Transactions serve as the initial step for all actions within the Ethereum system. They act as the "triggers" that prompt the Ethereum Virtual Machine to process contracts, update balances, and make changes to the Ethereum blockchain's overall state.

Exploring Ethereum Wallets: A Comprehensive Guide

Sahil Sojitra — Mon, 10 Jul 2023 12:29:20 +0000

Ethereum wallets serve as essential software applications that allow users to interact with Ethereum. They provide access to your funds, handle key management, track balances, and enable the creation and signing of transactions. Some wallets even support the interaction with smart contracts and tokens.

From a programmer's perspective, a wallet specifically refers to the system responsible for storing and managing a user's keys. While certain wallets solely focus on key management, others fall under the broader category of browsers, which act as interfaces for Ethereum-based decentralized applications (DApps).

In this chapter, we will primarily explore wallets as containers for private keys and systems designed to manage these keys.

Wallet Technology Overview

When designing Ethereum wallets, it's important to find a balance between convenience and privacy. Using a single key for everything is easy but compromises privacy, as all transactions can be traced. On the other hand, using a new key for each transaction is more private but difficult to manage. Good wallet design aims to strike the right balance.

Contrary to popular belief, Ethereum wallets only store keys, not actual ether or tokens. The tokens are recorded on the Ethereum blockchain, and users control them by signing transactions with their wallet keys. Think of an Ethereum wallet as a keychain. Although there is a technical distinction, it doesn't have a significant impact in practice. However, it's important to shift from a centralized banking mindset to a decentralized blockchain system, where anyone can see the balance but needs permission to initiate transactions.

Ethereum wallets are designed to hold cryptographic keys instead of directly storing ether or tokens. These wallets can be compared to keychains, as they house pairs of private and public keys. By utilizing their private keys, users can sign transactions, providing evidence of their ownership of the ether. The actual storage of ether occurs on the Ethereum blockchain.

There are two main types of wallets: nondeterministic and deterministic.

Nondeterministic wallets, also known as JBOK wallets, generate independent keys from random numbers. These keys are not related to each other.
Deterministic wallets derive all keys from a single master key called the seed. This means that all keys in a deterministic wallet are connected and can be regenerated if you have the original seed. Hierarchical Deterministic Wallets (BIP-32/BIP-44) use a tree-like structure for key derivation, which is commonly employed.

To enhance security for deterministic wallets, the seeds are often converted into a list of words known as mnemonic code words. This list should be written down and kept in a safe place, as it can be used to recreate the wallet if needed. However, it's crucial to protect the mnemonic code words, as anyone with access to them can gain control of the wallet and its associated ether and smart contracts. Avoid storing the recovery words electronically and instead write them on paper, storing them securely.

In the following sections, we will explore these wallet technologies in more detail. Remember to prioritize the security of your mnemonic code words!

Nondeterministic (Random) Wallets

The first Ethereum wallet, used during the pre-sale, stored a single randomly generated private key in each wallet file. However, these "old-style" wallets are being replaced by deterministic wallets, which are considered superior in many ways. It is recommended to avoid reusing Ethereum addresses to maximize privacy. This means using a new address (with a new private key) each time you receive funds. Some people even use a new address for each transaction, although this can become costly when dealing with tokens.

In a nondeterministic wallet, you need to regularly increase the list of keys and make backups. If you lose your data before backing up your wallet, you will lose access to your funds and smart contracts. The "type 0" nondeterministic wallets are particularly challenging because they create new wallet files for every new address.

Many Ethereum clients, like geth, use a keystore file. This file is JSON-encoded and contains a single randomly generated private key, encrypted with a passphrase for added security.

{
    "address": "001d3f1ef827552ae1114027bd3ecf1f086ba0f9",
    "crypto": {
        "cipher": "aes-128-ctr",
        "ciphertext":
            "233a9f4d236ed0c13394b504b6da5df02587c8bf1ad8946f6f2b58f055507ece",
        "cipherparams": {
            "iv": "d10c6ec5bae81b6cb9144de81037fa15"
        },
        "kdf": "scrypt",
        "kdfparams": {
            "dklen": 32,
            "n": 262144,
            "p": 1,
            "r": 8,
            "salt":
                "99d37a47c7c9429c66976f643f386a61b78b97f3246adca89abe4245d2788407"
        },
        "mac": "594c8df1c8ee0ded8255a50caf07e8c12061fd859f4b7c76ab704b17c957e842"
    },
    "id": "4fcb2ba4-ccdb-424f-89d5-26cce304bf9c",
    "version": 3
}

The keystore format employs a key derivation function (KDF), also known as a password stretching algorithm. It protects against brute-force, dictionary, and rainbow table attacks. The passphrase is repeatedly hashed for 262,144 rounds, making it difficult for attackers to guess the passphrase.

Several software libraries, such as keythereum, can read and write the keystore format.

Deterministic (Seeded) Wallets

Deterministic wallets, also known as seeded wallets, are wallets that have private keys derived from a single master key or seed. The seed is a randomly generated number combined with other data, like an index number or chain code. With a deterministic wallet, the seed alone is enough to recover all the derived keys. This means that a single backup taken during wallet creation is sufficient to secure all the funds and smart contracts within the wallet. The seed can also be used for wallet export or import, making it easy to transfer keys between different wallet systems.

The security of the seed is crucial because having access to the seed grants full access to the entire wallet. However, focusing security efforts on a single piece of data can be advantageous.

Hierarchical Deterministic Wallets (BIP-32/BIP-44)

Deterministic wallets were created to make it easier to generate multiple keys from a single seed. The most advanced type is called hierarchical deterministic (HD) wallets. In an HD wallet, keys are organized in a tree structure, with parent keys deriving child keys, which can then derive grandchild keys, and so on. This structure allows for better organization and management of keys.

HD wallets have two important advantages. First, they provide additional organizational meaning. For example, specific branches of keys can be designated for incoming payments or outgoing change. In corporate settings, different branches can be assigned to departments or accounting categories.

The second advantage is that HD wallets allow the creation of public keys without needing access to the private keys. This feature enables the use of HD wallets on insecure servers or in situations where only receiving or watching capabilities are needed.

In summary, HD wallets make it easier to generate and manage multiple keys from a single seed. They offer improved organization and the ability to create public keys without exposing private keys, enhancing security and flexibility in different scenarios.

Seeds and Mnemonic Codes (BIP-39)

One popular method for securely backing up and retrieving a private key is by using a sequence of words called a mnemonic. This approach, standardized by BIP-39, allows the private key to be recreated by arranging the words in the correct order. Many Ethereum wallets and other cryptocurrency wallets use this standard, making it easy to import and export seeds for backup and recovery using interoperable mnemonics.

To see why this approach has become popular, let’s have a look at an example:

A seed for a deterministic wallet, in hex

0c1e24e5917779d297e14d45f14e1a1a

A seed for a deterministic wallet, from a 12-word mnemonic

army van defense carry jealous true
garbage claim echo media make crunch

Writing down a long hex sequence for the private key can be prone to errors. On the other hand, using a list of known words is much easier because there is redundancy in word writing, especially in English. For example, if "inzect" was mistakenly recorded, it can be quickly identified during wallet recovery that "inzect" is not a valid English word and should be corrected to "insect". It is recommended to write down a representation of the seed as a backup for HD wallets since the seed is crucial for wallet recovery in case of data loss. However, the seed must be kept extremely private, so digital backups should be avoided. Writing it down on paper with a pen is the safest approach.

In summary, using a recovery word list to encode the seed for an HD wallet is the easiest and safest way to export, transcribe, record on paper, read without errors, and import a set of private keys into another wallet.

Wallet Best Practices

As cryptocurrency wallet technology has advanced, certain industry standards have emerged to ensure interoperability, ease of use, security, and flexibility. These standards allow wallets to generate keys for multiple cryptocurrencies using a single mnemonic. Here are the common standards:

Mnemonic code words: Based on BIP-39, these code words are used to create a mnemonic that represents the private keys. This standard enables easy backup and recovery of keys.
HD wallets: Based on BIP-32, HD wallets use a hierarchical structure to derive multiple keys from a single master key or seed. This provides efficient key management and organization.
Multipurpose HD wallet structure: Based on BIP-43, this standard extends HD wallets to support additional functionalities and purposes, enhancing their versatility.
Multicurrency and multiaccount wallets: Based on BIP-44, this standard allows wallets to manage multiple cryptocurrencies and multiple accounts within each cryptocurrency. It provides a unified approach to handling different currencies.

These standards have become the de facto wallet standard across various blockchain platforms. They are widely adopted by both software and hardware wallets, ensuring interoperability. Users can export a mnemonic generated by one wallet and import it into another wallet, recovering all their keys and addresses.

Examples of software wallets that support these standards include Jaxx, MetaMask, MyCrypto, and MyEtherWallet (MEW). Hardware wallets such as Keepkey, Ledger, and Trezor also adhere to these standards.

Mnemonic Code Words (BIP-39)

Mnemonic code words are word sequences that represent a random number used as a seed to generate a deterministic wallet. These words can be used to recreate the seed, and thus the entire wallet and its derived keys. When creating a wallet, users are typically presented with a sequence of 12 to 24 words as a backup. These words serve as a backup and can be used to recover the wallet in the same or compatible wallet applications. Mnemonic codes are defined in BIP-39, which is a widely adopted industry standard.

It's important to note that there are different implementations of mnemonic code standards. Electrum Bitcoin wallet, for example, uses a different set of words predating BIP-39 and is not compatible with it. However, BIP-39 has gained broad industry support and is considered the de facto standard. It allows for the creation of multicurrency wallets, including support for Ethereum, while Electrum seeds cannot.

The process of creating a mnemonic code and seed, as defined by BIP-39, involves nine steps. These steps are divided into two parts: generating the mnemonic words (steps 1-6) and converting the mnemonic to a seed (steps 7-9).

In summary, mnemonic code words are word sequences that represent a seed for deterministic wallets. They simplify wallet backup and recovery, and BIP-39 is the widely accepted industry standard for creating and utilizing mnemonic codes.

Generating mnemonic words

Mnemonic words are generated automatically by the wallet using the standardized process defined in BIP-39. The wallet starts from a source of entropy, adds a checksum, and then maps the entropy to a word list:

Create a cryptographically random sequence S of 128 to 256 bits.
Create a checksum of S by taking the first length-of-S ÷ 32 bits of the SHA-256 hash of S.
Add the checksum to the end of the random sequence S.
Divide the sequence-and-checksum concatenation into sections of 11 bits.
Map each 11-bit value to a word from the predefined dictionary of 2,048 words.
Create the mnemonic code from the sequence of words, maintaining the order.

Generating entropy and encoding as mnemonic words:

From Mnemonic to Seed

The mnemonic words represent a level of randomness, typically with a length of 128 to 256 bits. These words are then used to derive a longer 512-bit seed through a process called key-stretching. The key-stretching function used is called PBKDF2.

PBKDF2 takes two parameters: the mnemonic words and a salt. The salt serves two purposes. First, it adds extra complexity to prevent brute-force attacks. Second, it allows for the introduction of an optional passphrase, which adds an additional layer of security to protect the seed.

In the BIP-39 standard, the mnemonic words are passed as the first parameter to the PBKDF2 function, and the salt is formed by combining the constant string "mnemonic" with an optional user-supplied passphrase.

PBKDF2 then performs 2,048 rounds of hashing using the HMAC-SHA512 algorithm on the mnemonic and salt, resulting in a 512-bit value known as the seed.

Creating an HD Wallets from the Seed

HD wallets are built from a single root seed, which is a random number of 128, 256, or 512 bits. The root seed is typically generated from a mnemonic, as explained earlier.

Using this root seed, every key in the HD wallet is derived in a deterministic manner. This means that the entire HD wallet can be recreated from the root seed in any compatible HD wallet. This feature simplifies tasks such as exporting, backing up, restoring, and importing HD wallets that can contain numerous keys. Only the mnemonic, from which the root seed is derived, needs to be transferred for these operations.

HD Wallets (BIP-32) and Paths (BIP-43/44)

Most HD wallets adhere to the BIP-32 standard, which is widely accepted as the industry standard for deterministic key generation.

While we won't delve into all the intricacies of BIP-32, we'll cover the key components relevant for understanding its use in wallets. The important aspect to note is the hierarchical relationships that can exist among derived keys, as depicted in the "HD wallet: a tree of keys generated from a single seed" illustration. Additionally, it's essential to grasp the concepts of extended keys and hardened keys, which will be explained in the following sections.

There are numerous interoperable BIP-32 implementations available in various software libraries. Although most of these implementations are designed for Bitcoin wallets, they can still be used for Ethereum wallets by incorporating an Ethereum address encoding library or opting for a library specifically designed for Ethereum.

Extended Public and Private Keys

In BIP-32, keys can be extended to derive child keys, creating a hierarchical structure. This allows for the generation of a chain of keys and addresses. Parent keys can be extended by appending a chain code to them, which is a special 256-bit binary string mixed with each key to produce child keys.

If the key is a private key, it becomes an extended private key distinguished by the prefix xprv:

xprv9s21ZrQH143K2JF8RafpqtKiTbsbaxEeUaMnNHsm5o6wCW3z8ySyH4UxFVSfZ8n7ESu7fgir8i...

An extended public key is distinguished by the prefix xpub:

xpub661MyMwAqRbcEnKbXcCqD2GT1di5zQxVqoHPAgHNe8dv5JP8gWmDproS6kFHJnLZd23tWevhdn...

An important feature of HD wallets is the ability to derive child public keys from parent public keys without requiring access to the private keys. This means that child public keys can be derived either directly from the child private key or from the parent public key.

Using an extended public key, it is possible to derive all the public keys within a specific branch of the HD wallet structure. This enables the creation of highly secure deployments where a server or application holds an extended public key but does not possess any private keys. This allows for the generation of numerous public keys and Ethereum addresses, but prevents the ability to spend any funds sent to those addresses. The corresponding private keys can be securely stored on a separate server, ensuring that transactions can be signed and funds can be spent.

One practical use case for this approach is deploying an extended public key on a web server hosting an ecommerce application. The server can generate a new Ethereum address for each transaction (such as a customer's shopping cart) using the public key derivation function. Since the server does not have access to private keys, the risk of theft is minimized. This eliminates the need to generate and manage thousands of Ethereum addresses on a separate secure server, streamlining the process.

Another application is in cold-storage or hardware wallets. The extended private key can be stored in a hardware wallet for added security, while the extended public key can be kept online. Users can generate "receive" addresses as needed, while the private keys remain safely offline. When it's time to spend funds, the extended private key can be used in an offline signing Ethereum client or on the hardware wallet device to sign transactions.

Navigating the HD Wallet Tree Structure

The HD wallet tree structure offers great flexibility but can also become complex to navigate due to its unlimited potential for generations and children. To manage this complexity, two BIPs (Bitcoin Improvement Proposals) provide standards for structuring HD wallets.

BIP-43 suggests using the first hardened child index as a special identifier for the "purpose" of the tree structure. This means that an HD wallet should use only one branch of the tree, with the index number defining the purpose of the wallet. BIP-44 extends this idea to propose a multicurrency and multiaccount structure. In this structure, the purpose number is set to 44', and each currency has its own subtree under the second level.

BIP-44 specifies the structure as consisting of five predefined tree levels:

m / purpose' / coin_type' / account' / change / address_index

The HD wallet structure defined by BIP-44 has three levels: purpose', coin_type', and account'. The purpose' level is always set to 44'. The coin_type' level specifies the type of cryptocurrency coin, allowing for multiple currencies within the HD wallet. The account' level allows users to divide their wallets into separate logical subaccounts for organizational purposes.

In the Ethereum world, the fourth level of the path, change, has two subtrees: one for receiving addresses and one for change addresses. However, in Ethereum, only the "receive" path is used, as there is no need for change addresses like in Bitcoin. The address_index becomes the fifth level of the tree, and usable addresses are derived as children of this level.

The HD wallet path "m/44'/2'/0'/0/1" corresponds to the second private key in the Litecoin main account, specifically used for signing transactions.

Let's break down the path:

"m" indicates that it is an HD (hierarchical deterministic) wallet.
"44'" signifies the purpose level, which is set to 44' for BIP-44 standard compliance.
"2'" represents the coin type level, where 2' denotes Litecoin as the cryptocurrency.
"0'" indicates the account level, specifically the main account for Litecoin.
"0" represents the change level, which is not relevant in Litecoin as there are no separate change addresses.
"1" signifies the address index, specifically the second private key in the main account for Litecoin.

To summarize, the BIP-44 standard provides a structured approach for HD wallets, allowing for multicurrency and multi-account functionality. The purpose, coin type, account, and address index define the hierarchical levels within the wallet tree.

Conclusion

Wallets serve as the cornerstone of any blockchain application that interfaces with users. They provide the means for users to manage their assortment of keys and addresses. Additionally, wallets enable users to demonstrate their rightful ownership of ether and authorize transactions by applying digital signatures. Essentially, wallets act as the gateway for users to interact with the blockchain, ensuring secure verification of ownership and authorization of transactions.

Demystifying Public Key Cryptography: Exploring Ethereum, Digital Signatures, and Wallet Management

Sahil Sojitra — Fri, 07 Jul 2023 11:30:15 +0000

Ethereum relies on a powerful technology called cryptography, which uses math and computer security to keep things safe. Cryptography is more than just secret writing (encryption). It can prove knowledge of a secret without revealing it and ensure data is genuine. These tools are essential for Ethereum and other blockchain systems, and they are widely used in Ethereum applications.

Cryptography provides mathematical tools called cryptographic proofs. These tools allow us to show we know a secret without telling anyone what it is. For example, a digital signature can prove we own a secret key without showing the key itself. Digital fingerprints, called hashes, help us check if data is real and hasn't been tampered with.

Contrary to what you might think, Ethereum doesn't use encryption right now. This means that all communications and transactions in Ethereum are readable by everyone. It's done this way to make sure everyone can verify what's happening and reach an agreement. However, in the future, more advanced cryptographic tools may be used to encrypt some calculations while still keeping everyone on the same page.

Public key cryptography (PKC) is crucial in Ethereum for controlling who owns funds. It uses a pair of keys: a public key for encrypting and a private key for decrypting. We'll go over the basics of PKC and understand how it helps with ownership in Ethereum.

Now that we've explored the foundations of cryptography in Ethereum, we're ready to uncover more practical aspects. In the next post, we'll dive into wallets and see how they help manage keys effectively. This way, we can fully embrace Ethereum's decentralized world.

Keys and Addresses

In Ethereum, there are two types of accounts: externally owned accounts (EOAs) and contracts. EOAs are owned by individuals and are linked to digital private keys, Ethereum addresses, and digital signatures. Private keys are essential for interacting with Ethereum, as they uniquely determine Ethereum addresses. It's important to note that private keys should be kept private and never shared or stored on the Ethereum network.

Accessing and Controlling Funds

Digital signatures, created using private keys, play a crucial role in Ethereum transactions. A valid digital signature is required for a transaction to be included in the blockchain. Control over an account and its funds is granted to anyone who possesses the private key. Digital signatures in Ethereum transactions prove ownership of funds by demonstrating ownership of the private key.

Public and Private Keys

In Ethereum, keys come in pairs: a private (secret) key and a public key. The public key is similar to a bank account number, while the private key acts as a secret PIN, providing control over the account. Ethereum users rarely encounter private keys directly, as they are typically stored in encrypted files managed by Ethereum wallet software.

Ethereum Addresses

In Ethereum transactions, the recipient is represented by an Ethereum address, similar to the beneficiary account details in a bank transfer. Ethereum addresses for EOAs are generated from the public key portion of a key pair. It's worth noting that not all Ethereum addresses represent public-private key pairs; some can represent contracts, which don't have private keys.

Public Key Cryptography and Cryptocurrency

Public key cryptography (also called "asymmetric cryptography") is a core part of modern-day information security. The key exchange protocol, first published in the 1970s by Martin Hellman, Whitfield Diffie, and Ralph Merkle, was a monumental breakthrough that incited the first big wave of public interest in the field of cryptography

The Power of Mathematical Functions

Public key cryptography relies on mathematical functions with unique properties. These functions are easy to compute, but their inverse calculation is extremely challenging. For example, multiplying two large prime numbers together is simple, but finding the prime factors of the product (prime factorization) is extremely difficult.

Trapdoor Functions and Secrets

Certain mathematical functions can be easily reversed if you possess secret information. If I tell you that one of the prime factors is 2,003, you can trivially find the other one with a simple division: 8,018,009 ÷ 2,003 = 4,003. Such functions are often called trapdoor functions because they are very difficult to invert unless you are given a piece of secret information that can be used as a shortcut to reverse the function.

Elliptic Curve Cryptography

Elliptic curve arithmetic is a special kind of math used in cryptography. It involves doing calculations on a curved line. Multiplication on this line is easy, but division (finding the opposite) is really hard. This is called the discrete logarithm problem, and there aren't any known ways to solve it quickly. Because of this, elliptic curve cryptography is used a lot in modern computer systems. It's the basis for things like private keys and digital signatures in cryptocurrencies like Ethereum.

Public and Private Key Pair in Ethereum

In Ethereum, a public-private key pair is generated using public key cryptography. The private key is kept secret and controls access to an Ethereum account. The public key, derived from the private key, represents the account's address and allows for secure transactions and interactions with smart contracts.

The Role of Digital Signatures

A digital signature is like a special code used to sign messages. In Ethereum transactions, the transaction details are used as the message. Elliptic curve cryptography, a mathematical method, combines the transaction details with a private key to create a unique code called the digital signature. When sending a transaction on Ethereum to access an account or perform actions with smart contracts, it must be accompanied by a digital signature created using the private key associated with the Ethereum address.

Verification and Trust

The great thing about elliptic curve mathematics is that anyone can verify the validity of a transaction by checking if the digital signature matches the transaction details and Ethereum address. The verification process doesn't require knowing the private key; it remains private. However, the verification process ensures that the transaction could only come from someone who possesses the private key corresponding to the public key behind the Ethereum address. This is the "magic" of public key cryptography.

There is no encryption as part of the Ethereum protocol—all messages that are sent as part of the operation of the Ethereum network can (necessarily) be read by everyone. As such, private keys are only used to create digital signatures for transaction authentication.

Private Keys

A private key is a randomly chosen number that holds the utmost importance in Ethereum. It is the foundation of user control over funds associated with their Ethereum address and access to authorized contracts. The private key's primary role is to generate signatures required for spending ether, providing proof of ownership in transactions. It is crucial to keep the private key secret to prevent unauthorized access to the associated funds and contracts. Additionally, backing up and protecting the private key is essential to prevent permanent loss of funds in case of accidental misplacement.

The private key used in Ethereum is essentially a number. A simple method to generate private keys at random is by using a coin, pencil, and paper. By flipping a coin 256 times, you can obtain a sequence of binary digits that can be used as a random private key for an Ethereum wallet. From this private key, the corresponding public key and address can be derived.

Generating a Private Key

The process of generating a private key involves finding a secure source of randomness or entropy. The private key is essentially a number chosen between 1 and 2^256. The method used to select this number should be unpredictable and non-deterministic. Ethereum software relies on the random number generator of the underlying operating system, which is typically initialized by a human source of randomness. This may involve actions like moving the mouse or pressing random keys on the keyboard. Cosmic radiation noise on the computer's microphone channel can also serve as an alternative source of entropy.

The private key itself can be any nonzero number up to a massive value slightly less than 2^256. To create a private key, a 256-bit number is randomly chosen and checked for validity within this range. Typically, a larger string of random bits collected from a cryptographically secure source is fed into a 256-bit hash algorithm like Keccak-256 or SHA-256. If the resulting number falls within the valid range, it is considered a suitable private key. Otherwise, the process is repeated with another random number.

Important Considerations

Generating a private key is an offline process that doesn't require communication with the Ethereum network or anyone else. True randomness is crucial to ensure the uniqueness of the private key. Choosing a number yourself or using a weak random number generator increases the risk of someone else guessing the key and gaining access to your funds. Therefore, the private key should be truly random and unguessable. Remembering the private key is not necessary, allowing for the best approach in picking it: true randomness.

Public Key

An Ethereum public key represents a point on an elliptic curve, defined by a set of x and y coordinates that satisfy the equation of the curve.

In simpler terms, An Ethereum public key consists of two numbers that are derived from the private key through a one-way calculation. It is easy to calculate a public key from a private key, but it is impossible to calculate the private key from the public key.

Calculation and Irreversibility:

The public key is obtained by multiplying the private key with a constant point called the generator point using elliptic curve multiplication. This process is practically irreversible. The resulting public key (K) is calculated as K = k * G, where k is the private key, G is the generator point, and * represents the elliptic curve "multiplication" operator. It is important to note that this multiplication is different from regular integer arithmetic.

In simpler terms, To obtain the public key, the private key is multiplied by a generator point on the elliptic curve. However, there is no division operation, so it is not possible to calculate the private key by simply dividing the public key by the generator point. This irreversible mathematical function ensures the security of public key cryptography in Ethereum and other cryptocurrencies.

Elliptic curve multiplication is a "one-way" function used in cryptography. It's easy to perform multiplication, but impossible to reverse it through division. By using this function, the private key owner can generate a public key and share it, without worrying about anyone calculating the private key from the public key. This mathematical property enables the creation of secure digital signatures, proving ownership of Ethereum funds and control over contracts.

Elliptic Curve Cryptography Explained

Both Elliptic Curve Cryptography (ECC) and RSA serve the purpose of generating public and private keys for secure communication between parties. However, ECC offers certain advantages over RSA. For example, a 256-bit key in ECC provides similar security to a 3072-bit key in RSA. ECC also requires significantly less storage space and bandwidth compared to RSA, making it suitable for resource-constrained systems like smartphones, embedded computers, and cryptocurrency networks.

The Trapdoor Function in ECC:

One of the unique aspects of ECC is the trapdoor function, which sets it apart from RSA. Visualizing the trapdoor function can be likened to a mathematical game of pool.

Here's a simplified explanation of the algorithm:

Start at an arbitrary point on the curve (Point A).
Use the dot function to find a new point (Point B).
Repeat the dot function, hopping around the curve, until reaching the final point (Point E).

This algorithm follows a specific pattern:

A dot B = -C (Line from A to B intersects at -C).
Reflect across the X-axis from -C to C.
A dot C = -D (Line from A to C intersects at -D).
Reflect across the X-axis from -D to D.
A dot D = -E (Line from A to D intersects at -E).
Reflect across the X-axis from -E to E.

Answers To Some ECC's Questions

How is the second point determined? If the dot function involves connecting two points with a line, isn't it necessary to have an initial point and another point to begin with?

To find the second point in the elliptic curve arithmetic, we can perform a dot function operation. This operation involves drawing a line between two points. However, in this case, we don't need a separate second point to begin with.

Let's assume the first point is called P. The dot function of P with itself, denoted as P dot P, results in a second point, which we will call -R.

In other words, P dot P = -R.

What does P dot P represent? It is actually the tangent line of point P. This can be visualized in the following diagram:

So, by performing the dot function on a point P with itself, we obtain the tangent line at P, which gives us the second point -R.

If the number of hops represents the private key, wouldn't it be possible to iterate through the hops until reaching the endpoint by counting them?

No, it would not be feasible to simply count the hops until reaching the endpoint if the number of hops is extremely large, such as 2^256. The computational effort required to compute each hop individually, like p dot p dot p dot p..., would be impractical and time-consuming.

However, if you possess the knowledge of the specific number of hops, there is an alternative technique utilizing an exponentiation trick that enables a much faster determination of the endpoint. This method leverages the properties of elliptic curve operations. For instance, by using the concept of doubling, if you are aware that 2P is equivalent to P dot P, you can calculate 4P as 2P dot 2P. This approach allows for exponentially faster calculations even for exceedingly large values.

Cryptographic Hash Functions

Cryptographic hash functions play a crucial role in Ethereum and various cryptographic systems. They are widely recognized as the workhorses of modern cryptography due to their extensive use in securing data. In this section, we will explore the basic properties of hash functions and understand why they are so indispensable in Ethereum and other secure platforms. Hash functions are essential in transforming Ethereum public keys into addresses and creating digital fingerprints for data verification.

Understanding Hash Functions

In simple terms, a hash function is a special type of function that converts data of any size into a fixed-size output, known as the hash. Cryptographic hash functions, specifically, have specific properties that make them valuable for securing platforms like Ethereum.

Key Properties of Cryptographic Hash Functions

Determinism: A specific input always generates the same hash output.
Verifiability: Computing the hash of a message is efficient.
Noncorrelation: Even a minor change in the input data should result in an extensively different hash output, making it unrelated to the original message.
Irreversibility: It is practically impossible to compute the original message from its hash, requiring a brute-force search.
Collision protection: It should be extremely difficult to find two different messages that produce the same hash output. Hash collisions are exceptionally rare in Ethereum.

Applications and Uses of Cryptographic Hash Functions

Cryptographic hash functions serve various security purposes, including:

Data fingerprinting: Creating unique identifiers for data.
Message integrity: Detecting errors or tampering in messages.
Proof of work: Supporting consensus algorithms, like Ethereum's mining process.
Authentication: Protecting passwords and enhancing key security.
Pseudorandom number generation: Generating random values for cryptographic operations.
Message commitment: Ensuring commitment and later revealing of messages.
Unique identifiers: Assigning distinct labels to different entities.

Ethereum’s Cryptographic Hash Function: Keccak-256

Ethereum relies on the Keccak-256 cryptographic hash function for various purposes. Keccak-256 was chosen as the winner of the SHA-3 Cryptographic Hash Function Competition conducted by the National Institute of Standards and Technology (NIST) in 2007. It later became standardized as Federal Information Processing Standard (FIPS) 202 in 2015.

Keccak-256 and the NIST Controversy

During Ethereum's development, NIST was in the process of finalizing the standardization of Keccak-256. However, concerns arose regarding NIST's potential modifications to Keccak-256 due to revelations made by whistleblower Edward Snowden. These revelations suggested that the National Security Agency (NSA) had influenced NIST's random-number generator standard, raising doubts about its integrity.

The Ethereum Foundation's Decision

Given the controversy and uncertainty surrounding NIST's modifications, the standardization of SHA-3 experienced delays. Consequently, the Ethereum Foundation made the decision to implement the original Keccak algorithm proposed by its inventors, rather than using the modified SHA-3 standard endorsed by NIST. This choice aimed to ensure the security and reliability of Ethereum's cryptographic operations.

While Ethereum documents and code may mention "SHA-3," they often refer to a specific variant called Keccak-256 instead of the officially standardized FIPS-202 SHA-3. The slight implementation differences mainly involve padding parameters. However, these distinctions are significant because Keccak-256 and FIPS-202 SHA-3 produce different hash outputs for the same input.

Ethereum Addresses

Ethereum addresses are unique identifiers in the form of hexadecimal numbers. They are derived from the Keccak-256 hash of the public key, specifically the last 20 bytes.

Difference from Bitcoin Addresses

Unlike Bitcoin addresses, Ethereum addresses lack built-in checksums in their user interfaces. The reason behind this choice was the expectation that higher layers of the Ethereum system would handle checksums if needed, as addresses would be abstracted through name services.

Challenges Faced

However, the development of these higher layers progressed slower than expected. This led to issues in the early days of Ethereum, including funds lost due to mistyped addresses and input validation errors. Additionally, the adoption of alternative encodings by wallet developers was slow due to the delayed implementation of Ethereum name services.

Inter Exchange Client Address Protocol (ICAP)

The Inter Exchange Client Address Protocol (ICAP) is a method for encoding Ethereum addresses that offers convenience, error-checking, and compatibility with the International Bank Account Number (IBAN) format. ICAP provides a flexible and interoperable way to represent Ethereum addresses and registered names.

IBAN and ICAP

IBAN is a widely used standard for identifying bank account numbers in international wire transfers. ICAP serves as a decentralized alternative to IBAN, specifically designed for Ethereum addresses.

Structure and Components

Similar to IBAN, ICAP uses alphanumeric characters (letters and numbers), with a maximum length of 34 characters. It introduces a special country code, "XE," which stands for "Ethereum." ICAP includes a two-character checksum and three variations of an account identifier.

ICAP Variations:

Direct: This encoding represents the 155 least significant bits of an Ethereum address. It can handle addresses starting with zero bytes. It aligns with IBAN requirements but has limitations in representing the complete Ethereum address. Example: XE60HAMICDXSV5QXVJA7TJW47Q9CHWKJD (33 characters).
Basic: Similar to the Direct encoding, but allows representation of any Ethereum address. However, it is not compatible with IBAN field validation. Example: XE18CHDJBPLTBCJ03FE9O2NS0BPOJVQCU2P (35 characters).
Indirect: This variation encodes an identifier that links to an Ethereum address through a name registry provider. It includes an asset identifier, a name service, and a human-readable name. Example: XE##ETHXREGKITTYCATS (20 characters), where the "##" should be replaced by the computed checksum characters.

Hex Encoding with Checksum in Capitalization (EIP-55)

Ethereum Improvement Proposal 55 (EIP-55) offers a solution to enhance the accuracy of Ethereum addresses by introducing a checksum. This proposal addresses the slow implementation of ICAP and name services, providing a simple and effective way to prevent errors when working with Ethereum addresses.

EIP-55 Checksum

To ensure the integrity of Ethereum addresses, EIP-55 modifies the capitalization of certain characters in the address. By changing the capitalization, a checksum is embedded within the address itself. This checksum acts as a safeguard against mistakes during address input or reading.

Implementation and Benefits

Wallets and applications that support EIP-55 can verify the checksum within an Ethereum address. This enables them to detect and alert users about potential errors, improving address accuracy. However, wallets that do not support EIP-55 will still accept and process addresses without considering the checksum.

EIP-55 is quite simple to implement. We take the Keccak-256 hash of the lowercase hexadecimal address. This hash acts as a digital fingerprint of the address, giving us a convenient checksum. Any small change in the input (the address) should cause a big change in the resulting hash (the checksum), allowing us to detect errors effectively. The hash of our address is then encoded in the capitalization of the address itself. Let’s break it down, step by step:

Hash the lowercase address, without the 0x prefix:

Keccak256("001d3f1ef827552ae1114027bd3ecf1f086ba0f9") = 23a69c1653e4ebbb619b0b2cb8a9bad49892a8b9695d9a19d8f673ca991deae1

Capitalize each alphabetic address character if the corresponding hex digit of the hash is greater than or equal to 0x8. This is easier to show if we line up the address and the hash:

Address: 001d3f1ef827552ae1114027bd3ecf1f086ba0f9
Hash   : 23a69c1653e4ebbb619b0b2cb8a9bad49892a8b9...

In our address, there is a letter 'd' in the fourth position. We compare it to the fourth character of the hash, which is the number 6. Since 6 is less than 8, we keep the 'd' in lowercase. Moving on, the sixth character in our address is the letter 'f'. We compare it to the sixth character of the hash, which is the letter 'c'. Since 'c' is greater than 8, we change the 'f' in our address to uppercase 'F'. We continue this process, making changes based on the comparison with the corresponding characters in the hash. It's important to note that we only use the first 20 bytes (40 characters) of the hash as a checksum because our address can only accommodate 20 bytes (40 characters) for the appropriate capitalization.

Check the resulting mixed-capitals address yourself and see if you can tell which characters were capitalized and which characters they correspond to in the address hash:

Address: 001d3F1ef827552Ae1114027BD3ECF1f086bA0F9
Hash   : 23a69c1653e4ebbb619b0b2cb8a9bad49892a8b9...

Detecting an error in an EIP-55 encoded address

Now, let’s look at how EIP-55 addresses will help us find an error. Let’s assume we have printed out an Ethereum address, which is EIP-55 encoded:

0x001d3F1ef827552Ae1114027BD3ECF1f086bA0F9

Now let’s make a basic mistake in reading that address. The character before the last one is a capital F. For this example let’s assume we misread that as a capital E, and we type the following (incorrect) address into our wallet:

0x001d3F1ef827552Ae1114027BD3ECF1f086bA0E9

As you can see, even though the address has only changed by one character (in fact, only one bit, as e and f are one bit apart), the hash of the address has changed radically. That’s the property of hash functions that makes them so useful for checksums!

Now, let’s line up the two and check the capitalization:

001d3F1ef827552Ae1114027BD3ECF1f086bA0E9
5429b5d9460122fb4b11af9cb88b7bb76d892886...

It’s all wrong! Several of the alphabetic characters are incorrectly capitalized. Remember that the capitalization is the encoding of the correct checksum.

The capitalization of the address we input doesn’t match the checksum just calculated, meaning something has changed in the address, and an error has been introduced.

Essential Keywords Every Ethereum Enthusiast Must Know

Sahil Sojitra — Sat, 10 Jun 2023 13:27:29 +0000

Ethereum has emerged as one of the most influential platforms in the blockchain ecosystem. It's decentralized nature and smart contract capabilities have revolutionized various industries. However, before diving into Ethereum, it's essential to familiarize yourself with key keywords and concepts that form the foundation of this innovative technology.

Ethereum:

The keyword "Ethereum" refers to a decentralized blockchain platform and cryptocurrency that allows the creation and execution of smart contracts and decentralized applications (DApps).

It was proposed by Vitalik Buterin in 2013 and launched in 2015. Ethereum differs from Bitcoin, another popular cryptocurrency, in that it is not solely focused on financial transactions. Instead, it provides a more** general-purpose platform** for executing programmable contracts and applications.

Example:

A real-time example of Ethereum in action is the development of decentralized finance (DeFi) applications. DeFi applications aim to provide financial services and products in a decentralized manner, eliminating the need for traditional intermediaries such as banks.

One popular DeFi application built on Ethereum is Uniswap. Uniswap is a decentralized exchange protocol that allows users to trade ERC-20 tokens directly from their Ethereum wallets. It utilizes smart contracts to enable trustless and automated token swaps, providing liquidity through liquidity pools.

When users interact with Uniswap, they pay transaction fees in ETH to execute trades and interact with the smart contract. The decentralized nature of Uniswap ensures that users have full control over their funds and can participate in trading activities without the need for a centralized exchange.

Blockchain:

A blockchain is a distributed database that consists of a chain of blocks, where each block contains a set of transactions or other data. These blocks are linked together in chronological order, forming a chain. Each block contains a unique identifier (hash) and a reference to the previous block, which ensures the integrity and immutability of the data stored in the blockchain.

The decentralized nature of a blockchain means that it is maintained and updated by a network of computers (nodes) rather than a single central entity. Transactions are validated and added to the blockchain through a consensus mechanism, where participants in the network collectively agree on the validity of the transactions.

Blockchain technology provides several key features, including transparency, security, immutability, and decentralization.

Example:

In the Bitcoin blockchain, each block contains a set of transactions. When a user initiates a Bitcoin transaction, it is broadcasted to the network of nodes. Miners, who are participants in the network, compete to solve a complex mathematical puzzle to validate and add a new block of transactions to the blockchain. This process is known as mining. Once a block is successfully mined and added to the blockchain, the transaction becomes part of the public ledger and is considered confirmed.

The decentralized nature of the Bitcoin blockchain ensures that no single entity has control over the network. Transactions are transparently recorded on the blockchain, allowing users to verify the history and ownership of Bitcoin funds. The security of the blockchain is maintained through cryptographic algorithms, making it difficult for malicious actors to tamper with the transaction data.

Smart Contracts:

Smart contracts are self-executing contracts with the terms of the agreement directly written into code. They automatically execute predefined actions once the conditions specified within them are met. They are designed to facilitate, verify, or enforce the negotiation or performance of a contract. Smart contracts eliminate the need for intermediaries and enable parties to interact directly, securely, and transparently.

Smart contracts are typically written in programming languages specifically designed for the blockchain, such as Solidity for Ethereum. They define the rules and conditions of a contract, and once deployed on the blockchain, they become immutable and self-executing.

Smart contracts have access to the blockchain's decentralized infrastructure and can interact with other smart contracts and blockchain data. They can receive input, process information, and automatically execute actions based on predefined conditions. Smart contracts can be triggered by external events or by other smart contracts.

Example:

A real-time example is a decentralized crowdfunding platform on Ethereum, where a smart contract is created to handle the campaign. Participants send funds directly to the smart contract, which keeps track of the amount raised. The contract executes actions like transferring funds to the creator if the funding goal is reached within the specified timeframe or refunding the funds if the goal is not met. Smart contracts eliminate the need for intermediaries, ensuring transparency and trust in the execution of agreements.

Ethereum Network:

The Ethereum network refers to the collection of nodes (computers) running Ethereum software and participating in the validation and propagation of transactions and smart contracts. It encompasses various networks, including the mainnet (production network), testnets (Ropsten, Rinkeby, and others), and private networks for development and testing purposes.

Example:

An example of the Ethereum network is the popular decentralized application (DApp) called CryptoKitties. CryptoKitties is a game built on the Ethereum blockchain where users can buy, breed, and trade unique virtual cats.

When users interact with the CryptoKitties DApp, their actions are executed on the Ethereum network. For instance, when a user purchases a CryptoKitty, a transaction is initiated and broadcasted to the Ethereum network. Miners on the network validate and process the transaction, ensuring its correctness and security. Once the transaction is confirmed and added to a block, the ownership of the purchased CryptoKitty is updated on the blockchain.

The Ethereum network handles the validation, execution, and propagation of all CryptoKitties-related transactions, providing a secure and decentralized environment for users to interact with the game. It ensures that the ownership of CryptoKitties is accurately recorded and that the transactions are transparently executed, without the need for a central authority.

Ethereum Virtual Machine (EVM):

The Ethereum Virtual Machine (EVM) is a runtime environment within the Ethereum blockchain network. It executes smart contracts, which are self-executing agreements with predefined rules written in code. The EVM ensures the consistent and secure execution of these smart contracts across the network, allowing for the development of decentralized applications (DApps) on Ethereum. It uses the concept of "gas" to prioritize and secure operations within the network

The EVM's role is to execute the smart contract's logic, enforce the rules, and maintain the integrity and consistency of the decentralized DApp. It provides a secure and deterministic runtime environment for the execution of smart contracts on the Ethereum blockchain.

Peer-to-Peer Network (P2P):

A peer-to-peer (P2P) network is a type of decentralized network where participating devices, called peers, communicate and share resources directly with each other without the need for a central server. In a P2P network, each peer can act as both a client and a server, contributing resources and requesting services from other peers.

Here are some key characteristics and features of a peer-to-peer network:

Decentralization: P2P networks distribute the workload across multiple peers, eliminating the need for a central authority or server. Peers interact with each other on an equal basis, contributing and consuming resources.
Resource Sharing: Peers in a P2P network can share various resources, such as files, bandwidth, processing power, or services. Each peer contributes its resources to the network and can request resources from other peers.
Autonomy and Redundancy: P2P networks are resilient as there is no single point of failure. If one peer becomes unavailable or leaves the network, other peers can continue to operate and provide services independently.
Scalability: P2P networks can scale efficiently as the addition of new peers increases the available resources in the network. This makes them suitable for large-scale distributed applications or file-sharing systems.
Self-Organization: Peers in a P2P network discover and connect autonomously without relying on a central directory or server. They use protocols like Distributed Hash Tables (DHT) or Peer Discovery Protocols to locate and communicate with other peers.

Node:

A node in the Ethereum network is a computer or device that runs Ethereum software and maintains a copy of the blockchain. It participates in the validation and propagation of transactions, smart contracts, and other network activities. Nodes communicate with each other to synchronize data and ensure the consensus of the network.

There are two main types of nodes:

Full Node: A full node stores a complete copy of the Ethereum blockchain. It validates and verifies each transaction and smart contract independently. Full nodes help maintain the security and integrity of the network by independently verifying the blockchain's history. They also contribute to the decentralization of the network by participating in the consensus process.
Light Node: A light node, also known as a "light client," does not store a complete copy of the blockchain. Instead, it relies on full nodes for blockchain data and validation. Light nodes consume less storage space and bandwidth compared to full nodes but sacrifice some degree of decentralization and security

Example:

A real-time example of a node in the Ethereum network is a user's Ethereum wallet or a dApp browser like MetaMask. These applications connect to the Ethereum network through nodes. For instance, when a user accesses their Ethereum wallet using MetaMask, it connects to a selected node.

The node provides the necessary blockchain data for the wallet to function. Users can view their account balance, send transactions, and interact with smart contracts through the wallet interface. The node facilitates communication between the wallet application and the Ethereum blockchain, ensuring that user actions are executed and responses from the blockchain are delivered back to the wallet.

Miners:

Miners are individuals or entities in a blockchain network who perform the process of mining. Mining involves solving complex mathematical puzzles or algorithms to validate and confirm transactions. Miners compete with each other to find the solution to the puzzle, and the first miner to solve it earns the right to add a new block of transactions to the blockchain.

When a miner successfully solves the puzzle, they create a new block that includes a set of validated transactions. This block is then added to the existing blockchain, forming a continuous chain of blocks. The process of mining not only validates transactions but also secures the network against fraudulent activities, as altering past blocks becomes computationally impractical due to the computational power required to do so.

Miners are rewarded for their efforts in the form of incentives. In some blockchain networks, such as Ethereum, miners are rewarded with cryptocurrency tokens. For example, in Ethereum, miners receive Ether (ETH) as a reward for successfully mining a block. The mining process also involves transaction fees paid by users for their transactions to be prioritized and included in the blocks.

Example:

A real-time example of miners can be seen in the Bitcoin network. When a user initiates a Bitcoin transaction, it is broadcasted to the network, and miners collect these transactions to create a candidate block. Miners then compete to find a solution to the mathematical puzzle associated with mining, and the first miner to solve it adds the candidate block to the blockchain.

In the Bitcoin network, miners earn newly minted Bitcoins and transaction fees as rewards for successfully mining a block. Their computational efforts ensure the validation of transactions and the overall security of the blockchain network. This process demonstrates the essential role of miners in maintaining a robust and trustworthy decentralized system.

Gas:

Gas refers to the unit of measurement for the computational effort required to execute operations or transactions on a blockchain network. It is primarily associated with the Ethereum platform. Gas is used to quantify the resources consumed during the execution of smart contracts and helps determine the fees required to process transactions.

Example:

In Ethereum, when a user wants to execute a transaction or interact with a smart contract, they need to specify the amount of gas they are willing to use and the price they are willing to pay for each unit of gas.

For instance, let's say a user wants to transfer tokens using a smart contract. They estimate that the transaction will require 50,000 gas units. They set a gas price of 10 Gwei (0.00000001 ETH) per unit of gas and a gas limit of 100,000 gas units.

The gas price is multiplied by the gas units to calculate the transaction fee. In this case, it would be 0.0005 ETH. As the transaction is executed, the smart contract operations consume gas, deducting it from the gas limit. If the gas limit is reached before the transaction completes, it fails, and the changes are rolled back.

Execution Client:

An execution client is a software application that serves as a bridge between users and the Ethereum network. It enables users to interact with smart contracts, send transactions, and retrieve blockchain data. Execution clients handle tasks such as signing transactions, managing account information, and communicating with the Ethereum network.

There are different types of execution clients available for Ethereum, including Geth, Parity, and OpenEthereum. These clients implement the Ethereum protocol and provide interfaces for users to interact with the blockchain.

Example:

A real-time example of an execution client is the MetaMask wallet. MetaMask is a popular browser extension that acts as an execution client and provides a user-friendly interface for interacting with the Ethereum network.

When a user installs MetaMask and connects it to their Ethereum wallet, it becomes an execution client. Users can then access decentralized applications (DApps) directly from their web browsers. MetaMask allows users to interact with smart contracts, send and receive transactions, and manage their Ethereum accounts.

For instance, if a user wants to participate in a decentralized finance (DeFi) application, they can connect MetaMask to the application. MetaMask serves as the execution client, handling the transaction signing process and communicating with the Ethereum network. Users can then execute various actions within the DeFi application, such as depositing funds, swapping tokens, or providing liquidity.

Consensus Client:

A consensus client is responsible for validating and agreeing upon the transactions and the order in which they are added to the blockchain. It ensures that all network participants reach a consensus on the state of the blockchain, meaning they agree on the validity and ordering of transactions.

Consensus clients typically implement a specific consensus algorithm, such as Proof-of-Work (PoW), Proof-of-Stake (PoS), or other variants. These algorithms determine how consensus is achieved in the network, considering factors like computational power, token ownership, or reputation.

Example:

A real-time example of a consensus client is the Geth client in the Ethereum network. Geth is one of the most widely used clients that implement the Ethereum protocol and the consensus algorithm.

When a user runs Geth as a consensus client, it connects to the Ethereum network and participates in the consensus process. Geth verifies the validity of transactions, executes smart contracts, and contributes to the consensus mechanism of the network.

For example, when a user submits a transaction to the Ethereum network, Geth receives and validates it. Geth then propagates the transaction to other nodes, which also run consensus clients, and participates in the consensus algorithm to determine the transaction's validity and order.

In this example, Geth acts as a consensus client in the Ethereum network, helping to maintain a decentralized and secure blockchain by participating in the consensus process. It plays a vital role in reaching an agreement on the state of the Ethereum blockchain among network participants.

Proof-Of-Work (PoW):

Proof of Work (PoW) in Ethereum is a mechanism used to validate and secure transactions on the Ethereum blockchain. In simple terms, it is a way to ensure that transactions are legitimate and prevent malicious activities.

Here's a simplified explanation of how Proof of Work works in Ethereum:

Miners: In Ethereum, miners are participants who compete with each other to solve complex mathematical puzzles.
Puzzle-solving: Miners use their computational power to solve these puzzles, which require a significant amount of computational effort. The puzzles are designed in such a way that the only way to solve them is through trial and error, with no shortcuts available.
Validating transactions: Miners collect pending transactions from the network and include them in blocks.
Mining process: Miners bundle these transactions together in a block and start solving the puzzle associated with that block. They repeatedly generate different solutions by guessing a random number until one of them finds a solution that satisfies the puzzle's requirements.
Proof of solution: Once a miner finds a solution, they broadcast it to the network as proof that they have done the necessary work to validate the transactions in that block.
Consensus: Other participants in the network can quickly verify the validity of the solution provided by the winning miner. They check if the solution meets the required criteria and if the transactions in the block are valid. If everything is correct, the block is considered validated.
Block reward: The winning miner who successfully solves the puzzle and validates the block is rewarded with newly created Ether (the native cryptocurrency of Ethereum) and any transaction fees included in that block.
Difficulty adjustment: The Ethereum network automatically adjusts the difficulty of the puzzles based on the total computational power of the network. This adjustment ensures that new blocks are added to the blockchain approximately every 15 seconds on average, regardless of the total computational power of the network.

Example:

A popular example of PoW is Bitcoin mining. Miners compete to solve complex mathematical puzzles by guessing different nonces. The first miner who finds the correct nonce gets to add a new block of transactions to the Bitcoin blockchain and receives a reward in the form of newly created bitcoins.

The puzzles in Bitcoin mining involve finding a nonce that, when combined with the block data, produces a hash (a unique digital fingerprint) with a specific pattern. Miners use powerful computers to calculate trillions of possibilities per second until one of them discovers the correct nonce.

By solving these puzzles, miners secure the Bitcoin network and validate transactions. The computational work required to solve the puzzles makes it difficult for anyone to manipulate the blockchain, ensuring the system's integrity and reliability.

Proof-Of-Stake (PoS):

Proof of Stake (PoS) is an alternative consensus mechanism used in Ethereum to validate and secure transactions on the blockchain. Unlike Proof of Work, which relies on computational power, Proof of Stake uses a different approach that involves participants staking their cryptocurrency holdings to gain the right to validate transactions.

Here's a simplified explanation of how Proof of Stake works in Ethereum:

Validators: Instead of miners, Ethereum's Proof of Stake involves validators. Validators are participants who hold a certain amount of Ether (the native cryptocurrency of Ethereum) and are willing to lock up or "stake" their coins as collateral.
Block selection: Validators take turns being randomly chosen to create new blocks and validate transactions. The chances of being selected as a validator are based on the proportion of Ether they have staked compared to others in the network. The more Ether staked, the higher the chances of being selected.
Validation process: When a validator is selected, they create a new block and include transactions within it. Instead of solving complex puzzles like in Proof of Work, validators validate transactions based on the amount of Ether they have staked. This means that validators have an economic incentive to act honestly since any malicious behavior may result in their staked Ether being slashed or taken away.
Consensus and rewards: Once a block is created and validated, other validators in the network verify its correctness. If the block is deemed valid, it is added to the blockchain, and the validator who created the block is rewarded with transaction fees and additional Ether as a form of incentive for their participation.
Security and decentralization: Proof of Stake aims to secure the blockchain by ensuring that validators have a stake in the system. If a validator tries to act dishonestly or attack the network, they risk losing their staked Ether, which acts as a deterrent against malicious behavior. The more Ether staked in the network, the more decentralized and secure it becomes.
Finality: In Proof of Stake, transactions are finalized more quickly compared to Proof of Work. Once a block is added to the blockchain and a certain number of subsequent blocks are built on top of it, the transactions in that block are considered final and cannot be reversed.

Example:

An example of PoS in action is the Tezos blockchain network. Tezos utilizes PoS to secure its network and validate transactions. In Tezos, stakeholders who hold XTZ cryptocurrency can participate in the consensus process.

Stakeholders can choose to become "bakers" by locking up a certain amount of XTZ as collateral and participating in block validation. Bakers are randomly selected to create and validate new blocks based on the amount of XTZ they hold and are willing to stake.

The Tezos PoS system allows for more energy-efficient and environmentally friendly consensus compared to PoW systems like Bitcoin. It also promotes decentralization by giving stakeholders the opportunity to actively participate in securing the network and earning rewards.

Understanding essential keywords is crucial for Ethereum enthusiasts. Concepts such as smart contracts, the EVM, gas, nodes, miners, and execution clients provide a solid foundation for engaging with Ethereum. By grasping these fundamentals, enthusiasts can navigate decentralized applications, participate in consensus, and make informed decisions. As Ethereum continues to evolve, staying updated with the latest keywords is essential for maximizing involvement. Embrace the potential of this groundbreaking technology and continue the Ethereum journey with enthusiasm.

Happy learning and exploring Ethereum!

Mastering Go: Unleashing the Potential of Methods, Interfaces, Generics, and Concurrency | Part - 3

Sahil Sojitra — Sat, 27 May 2023 15:12:59 +0000

In the world of programming languages, Golang (also known as Go) has gained significant popularity for its simplicity, efficiency, and robustness. With its unique syntax and powerful features, Golang offers developers a versatile toolkit to build scalable and high-performance applications. In this blog, we will dive deep into some of the key aspects of Golang syntax, namely , Methods, Interfaces, Generics & Concurrency. Let's explore each topic in detail:

Methods

Go does not have classes. However, you can define methods on types.
A method is a function with a special receiver argument. = The receiver appears in its own argument list between the func keyword and the method name.
In this example, the Abs method has a receiver of type Vertex named v.

package main

import (
    "fmt"
    "math"
)

type Vertex struct {
    X, Y float64
}

// Remember: a method is just a function with a receiver argument.
// Here's Abs written as a regular function with no change in functionality.

func Abs_func(v Vertex) float64 {
    return math.Sqrt(v.X*v.X + v.Y*v.Y)
}

func (v Vertex) Abs() float64 {
    return math.Sqrt(v.X*v.X + v.Y*v.Y)
}

func main() {
    v := Vertex{3, 4}
    fmt.Println(v.Abs())

    f := Vertex{12, 5}
    fmt.Println(Abs_func(f))
}

Output:

5
13

You can only declare a method with a receiver whose type is defined in the same package as the method. You cannot declare a method with a receiver whose type is defined in another package (which includes the built-in types such as int).

package main

import (
    "fmt"
    "math"
)

type MyFloat float64

func (f MyFloat) Abs() float64 {
    if f < 0 {
        return float64(-f)
    }
    return float64(f)
}

func main() {
    f := MyFloat(-math.Sqrt(2))
    fmt.Println(f)
    fmt.Println(f.Abs())
}

Output

-1.4142135623730951
1.4142135623730951

Pointer Receivers

Go allows the declaration of methods with pointer receivers. Pointer receivers have the syntax *T, where T is a type (not a pointer type itself, such as *int). These methods can modify the value to which the receiver points. In contrast, methods with value receivers operate on a copy of the original value.
For example, consider the Scale method defined on *Vertex. If the * is removed from the receiver declaration, the method will no longer be able to modify the original Vertex value. Pointer receivers are commonly used when methods need to modify their receiver.
Removing the * from the receiver declaration changes the behavior to operate on a copy of the value instead.

package main 

import (
    "fmt"
    "math"
)

type Vertex struct {
    X, Y float64
}

func (v Vertex) Abs() float64 {
    return math.Sqrt(v.X*v.X + v.Y*v.Y)
}

// Method With Pointer Receiver
func (v *Vertex) Scale(f float64) {
    v.X = v.X * f
    v.Y = v.Y * f
    fmt.Println(v.X,v.Y)
}

// Scale Function
func (v Vertex) Scale_(f float64) {
    v.X = v.X * f
    v.Y = v.Y * f
    fmt.Println(v.X,v.Y)
}

func main() {
    v := Vertex{3, 4}
    v.Scale(10)
    fmt.Println(v.X, v.Y)
    fmt.Println(v.Abs(),"\n")

    f := Vertex{3, 4}
    f.Scale_(10)
    fmt.Println(f.X, f.Y)
    fmt.Println(f.Abs())
}

Output:

Methods and Pointer Indirection

you might notice that functions with a pointer argument must take a pointer:

var v Vertex
ScaleFunc(v, 5)  // Compile error!
ScaleFunc(&v, 5) // OK

while methods with pointer receivers take either a value or a pointer as the receiver when they are called:

var v Vertex
v.Scale(5)  // OK
p := &v
p.Scale(10) // OK

Consider the following example:

package main

import "fmt"

type Vertex struct {
    X, Y float64
}

func (v *Vertex) Scale(f float64) {
    v.X = v.X * f
    v.Y = v.Y * f
}

func ScaleFunc(v *Vertex, f float64) {
    v.X = v.X * f
    v.Y = v.Y * f
}

func main() {
    v := Vertex{3, 4}
    v.Scale(2)
    fmt.Println(v)
    ScaleFunc(&v, 10)
    fmt.Println(v,"\n")

    p := &Vertex{3, 4}
    p.Scale(2)
    fmt.Println(p)
    ScaleFunc(p,10)
    fmt.Println(p)
}

Output:

{6 8}
{60 80} 

&{6 8}
&{60 80}

The equivalent thing happens in the reverse direction.
Functions that take a value argument must take a value of that specific type:

var v Vertex
fmt.Println(AbsFunc(v))  // OK
fmt.Println(AbsFunc(&v)) // Compile error!

while methods with value receivers take either a value or a pointer as the receiver when they are called:

var v Vertex
fmt.Println(v.Abs()) // OK
p := &v
fmt.Println(p.Abs()) // OK

In this case, the method call p.Abs() is interpreted as (*p).Abs().

package main

import (
    "fmt"
    "math"
)

type Vertex struct {
    X, Y float64
}

func (v Vertex) Abs() float64 {
    return math.Sqrt(v.X*v.X + v.Y*v.Y)
}

func AbsFunc(v Vertex) float64 {
    return math.Sqrt(v.X*v.X + v.Y*v.Y)
}

func main() {
    v := Vertex{3, 4}
    fmt.Println(v.Abs())
    fmt.Println(AbsFunc(v),"\n")

    p := &Vertex{3, 4}
    fmt.Println(p.Abs())
    fmt.Println(AbsFunc(*p))
}

Output:

Choosing a value or pointer receiver

package main

import (
    "fmt"
    "math"
)

type Vertex struct {
    X, Y float64
}

func (v *Vertex) Scale(f float64) {
    v.X = v.X * f
    v.Y = v.Y * f
}

func (v *Vertex) Abs() float64 {
    return math.Sqrt(v.X*v.X + v.Y*v.Y)
}

func main() {
    v := &Vertex{3, 4}
    fmt.Printf("Before scaling: %+v, Abs: %v\n", v, v.Abs())
    v.Scale(5)
    fmt.Printf("After scaling: %+v, Abs: %v\n", v, v.Abs())
}

Output:

Before scaling: &{X:3 Y:4}, Abs: 5
After scaling: &{X:15 Y:20}, Abs: 25

Interfaces

In Go, interfaces provide a way to define sets of methods that a type must implement. This enables polymorphism and allows different types to be treated interchangeably if they satisfy the interface contract. Here's an example of using interfaces in Go:

package main

import (
    "fmt"
    "math"
)

// Shape is an interface for geometric shapes
type Shape interface {
    Area() float64
    Perimeter() float64
}

// Rectangle represents a rectangle shape
type Rectangle struct {
    Width  float64
    Height float64
}

// Area calculates the area of the rectangle
func (r Rectangle) Area() float64 {
    return r.Width * r.Height
}

// Perimeter calculates the perimeter of the rectangle
func (r Rectangle) Perimeter() float64 {
    return 2 * (r.Width + r.Height)
}

// Circle represents a circle shape
type Circle struct {
    Radius float64
}

// Area calculates the area of the circle
func (c Circle) Area() float64 {
    return math.Pi * c.Radius * c.Radius
}

// Perimeter calculates the circumference of the circle
func (c Circle) Perimeter() float64 {
    return 2 * math.Pi * c.Radius
}

func main() {
    rect := Rectangle{Width: 5, Height: 3}
    circle := Circle{Radius: 2.5}

    shapes := []Shape{rect, circle}

    for _, shape := range shapes {
        fmt.Printf("Area: %f\n", shape.Area())
        fmt.Printf("Perimeter: %f\n", shape.Perimeter())
        fmt.Println("------------------")
    }
}

Output:

Area: 15.000000
Perimeter: 16.000000
------------------
Area: 19.634954
Perimeter: 15.707963
------------------

Stringers

In Go, the Stringer interface is a built-in interface that allows types to define their own string representation. The String() method of the Stringer interface returns a string representation of the type. Here's an example of using the Stringer interface in Go:

package main

import "fmt"

type Person struct {
    Name string
    other string
}

func (p Person) String() string {
    return fmt.Sprintf("%v %v", p.Name, p.other)
}

func main() {
    a := Person{"Jay!", "Shiya Ram"}
    z := Person{"Jay Shree!", "Radhe Krishna"}
    fmt.Println(a, z)
}

Output:

Jay! Shiya Ram Jay Shree! Radhe Krishna

Type Parameters

Go functions can be written to work on multiple types using type parameters. The type parameters of a function appear between brackets, before the function's arguments.

func Index[T comparable](s []T, x T) int

This declaration means that s is a slice of any type T that fulfills the built-in constraint comparable. x is also a value of the same type.
comparable is a useful constraint that makes it possible to use the == and != operators on values of the type. In this example, we use it to compare a value to all slice elements until a match is found. This Index function works for any type that supports comparison.

package main

import "fmt"

// Index returns the index of x in s, or -1 if not found.
func Index[T comparable](s []T, x T) int {
    for i, v := range s {
        // v and x are type T, which has the comparable
        // constraint, so we can use == here.
        if v == x {
            return i
        }
    }
    return -1
}

func main() {
    // Index works on a slice of ints
    si := []int{10, 20, 15, -10}
    fmt.Println(Index(si, 15))

    // Index also works on a slice of strings
    ss := []string{"foo", "bar", "baz"}
    fmt.Println(Index(ss, "hello"))
}

Output:

2
-1

Goroutines

A goroutine is a lightweight thread managed by the Go runtime.

go f(x, y, z)

starts a new goroutine running

f(x, y, z)

The evaluation of f, x, y, and z happens in the current goroutine and the execution of f happens in the new goroutine.
Goroutines run in the same address space, so access to shared memory must be synchronized. The sync package provides useful primitives, although you won't need them much in Go as there are other primitives.

package main

import (
    "fmt"
    "time"
)

func say(s string) {
    for i := 0; i < 5; i++ {
        time.Sleep(100 * time.Millisecond)
        fmt.Println(s)
    }
}

func main() {
    go say("world")
    say("hello")
}

Output:

hello
world
world
hello
world
hello
hello
world
world
hello

In this code, we have two functions: say and main. The say function takes a string parameter s and prints it five times with a delay of 100 milliseconds between each print.
In the main function, we launch a new goroutine by calling go say("world"). This means that the say function with the argument "world" will be executed concurrently with the main goroutine.
Simultaneously, the main goroutine continues executing and calls say("hello"). As a result, "hello" will be printed five times in the main goroutine.
The output of this program will be somewhat unpredictable due to the concurrent nature of goroutines. It may vary on each execution, but you can expect to see interleaved "hello" and "world" messages.

Channels

channels provide a way for goroutines to communicate and synchronize their execution. Channels are used to pass data between goroutines and ensure safe concurrent access to shared resources. Here's an explanation of channels in Go:

1. Channel Creation:

To create a channel, you use the make function with the chan keyword followed by the type of data the channel will transmit. For example:

ch := make(chan int) // Creates an unbuffered channel of type int

2. Channel Operations:

Channels support two fundamental operations: sending and receiving data.

Sending Data: To send data through a channel, you use the <- operator in the form channel <- value. For example:

ch <- 177 // Sends the value 177 into the channel

Receiving Data: To receive data from a channel, you use the <- operator on the left-hand side of an assignment. For example:

value := <-ch // Receives a value from the channel and assigns it to the variable "value"

Consider the following example:

package main

import "fmt"

func sum(s []int, c chan int) {
    sum := 0
    for _, v := range s {
        sum += v
    }
    c <- sum // send sum to c
}

func main() {
    s := []int{7, 2, 8, -9, 4, 0}

    c := make(chan int)
    go sum(s[:len(s)/2], c)
    go sum(s[len(s)/2:], c)
    x, y := <-c, <-c // receive from c

    fmt.Println(x, y, x+y)
}

Output:

-5 17 12

The example code sums the numbers in a slice, distributing the work between two goroutines. Once both goroutines have completed their computation, it calculates the final result.

Buffered Channels

Channels can be buffered. Provide the buffer length as the second argument to make to initialize a buffered channel:

ch := make(chan int, 100)

Sends to a buffered channel block only when the buffer is full. Receives block when the buffer is empty.

package main

import (
    "fmt"
)

func main() {
    // Create a buffered channel with a capacity of 3
    ch := make(chan int, 3)

    // Send values to the channel
    ch <- 1
    ch <- 2
    ch <- 3

    // Attempting to send another value to the channel would block since the buffer is full

    // Receive values from the channel
    fmt.Println(<-ch)
    fmt.Println(<-ch)
    fmt.Println(<-ch)

    // Attempting to receive another value from the channel would block since the buffer is empty
}

Output:

1
2
3

In this example, we create a buffered channel ch with a capacity of 3 by specifying the capacity as the second argument to the make function.
We then send three values (1, 2, and 3) to the channel using the <- operator. Since the channel has a buffer capacity of 3, these sends will not block.
After sending the values, we receive and print them using the <- operator and fmt.Println() statements. Again, since the channel is buffered and contains three values, these receives will not block.
However, if we attempt to send or receive more values to/from the channel, it would block. For example, trying to send a value when the buffer is full or receive a value when the buffer is empty would cause the corresponding goroutine to block until space becomes available or a value is sent. -Buffered channels are useful when you want to decouple the send and receive operations in terms of timing, allowing the sender and receiver to operate independently up to the buffer capacity.

To continue reading and explore the other chapter, simply follow this link: Link To Other Chapters

Demystifying Go's Syntax: A Sarcastic Journey through Pointers, Structs, Arrays, Slices, and Maps | Part - 2

Sahil Sojitra — Sat, 27 May 2023 07:47:34 +0000

In the world of programming languages, Golang (also known as Go) has gained significant popularity for its simplicity, efficiency, and robustness. With its unique syntax and powerful features, Golang offers developers a versatile toolkit to build scalable and high-performance applications. In this blog, we will dive deep into some of the key aspects of Golang syntax, namely pointers, struct, arrays, slices & maps. Let's explore each topic in detail:

Pointers:

Pointers play a crucial role in Golang and understanding them is essential for effective memory management and manipulating data. This section will cover the basics of pointers, including their declaration, referencing, and dereferencing. We will explore how pointers enable pass-by-reference and allow us to modify variables directly in memory.

Go has pointers. A pointer holds the memory address of a value.
The type *T is a pointer to a T value. Its zero value is nil.

var p *int

The & operator generates a pointer to its operand.

i := 177
p = &i

The * operator denotes the pointer's underlying value.

fmt.Println(*p) // read i through the pointer p
*p = 146        // set i through the pointer p

This is known as "dereferencing" or "indirecting".

Consider the following example:

package main

import "fmt"

func main() {

    i, j := 146, 5256 // initialize the i,j 

    p := &i         // point to i
    fmt.Println(*p) // read i through the pointer
    *p = 177        // set i through the pointer
    fmt.Println(i)  // see the new value of i

    p = &j           // point to j
    *p = *p / 146   // divide j through the pointer
    fmt.Println(j) // see the new value of j
}

Output:

146
177
36

Structs

In Go, a struct is a composite data type that allows you to define your own custom data structure by grouping together different types of values into a single entity. It is similar to a class in object-oriented programming languages but without methods.

To define a struct in Go, you use the type keyword followed by the name of the struct and a set of fields enclosed in curly braces. Here's an example:

A struct is a collection of fields.
Struct fields are accessed using a dot.
Struct fields can be accessed through a struct pointer. To access the field X of a struct when we have the struct pointer p we could write (*p).X. However, that notation is cumbersome, so the language permits us instead to write just p.X, without the explicit dereference.
A struct literal denotes a newly allocated struct value by listing the values of its fields. You can list just a subset of fields by using the Name: syntax.

package main

import "fmt"

// Structs  
type Vertex struct {
    X int
    Y int
}

var (
    v1 = Vertex{1, 2}  // has type Vertex
    v2 = Vertex{X: 1}  // Y:0 is implicit
    v3 = Vertex{}      // X:0 and Y:0
    q  = &Vertex{1, 2} // has type *Vertex
)

func main() {

    // Struct
    fmt.Println(Vertex{1, 2})

    // Struct Fields
    v := Vertex{1, 2}
    v.X = 4
    fmt.Println(v.X,v.Y)

    // Pointers to Struct
    f := Vertex{1, 2}
    p := &f
    p.X = 3
    fmt.Println(f)

    // Struct Literals
    fmt.Println(v1,v2, v3, q)   
}

Output:

{1 2}
4 2
{3 2}
{1 2} {1 0} {0 0} &{1 2}

Arrays

In Go, an array is a fixed-size collection of elements of the same type. The size of an array is specified at compile time and cannot be changed during runtime. Here's an example of declaring and using an array:

package main

import "fmt"

func main() {
    var a [2]string
    a[0] = "Hello"
    a[1] = "World"
    fmt.Println(a[0], a[1])
    fmt.Println(a)

    primes := [6]int{2, 3, 5, 7, 11, 13}
    fmt.Println(primes)
}

Output:

Hello World
[Hello World]
[2 3 5 7 11 13]

Slices

Slices are dynamic, resizable, and more flexible than arrays in Go. They are built on top of arrays and provide additional functionality. Slices do not have a fixed size and can grow or shrink as needed. Here's an example:

package main

import "fmt"

func main() {
    primes := []int{2, 3, 5, 7, 11, 13}

    primes = append(primes, 17)   // Appending elements to the slice
    primes = append(primes, 19)
    primes = append(primes, 23)

    fmt.Println(primes)            // Output: [2 3 5 7 11 13 17 19 23]
    fmt.Println(primes[1])         // Output: 3
    fmt.Println(len(primes))       // Output: 9 (length of the slice)

    // Slicing a slice to create a new slice
    newSlice := primes[1:5]         // Contains elements at indices 1 and 4 (inclusive:exclusive)
    fmt.Println(newSlice)           // Output: [3 5 7 11]
}

Output:

[2 3 5 7 11 13 17 19 23]
3
9
[3 5 7 11]

Slices are like references to arrays

A slice does not store any data, it just describes a section of an underlying array.
Changing the elements of a slice modifies the corresponding elements of its underlying array.
Other slices that share the same underlying array will see those changes.

package main

import "fmt"

func main() {
    names := [4]string{
        "Ram",
        "Lakshman",
        "Parshuram",
        "Samvad",
    }
    fmt.Println(names)

    a := names[0:2] // 0 to 1
    b := names[1:3] // 1 to 2
    fmt.Println(a, b)

    b[0] = "Krishna"
    // Changing the element of a Slice b
    // Modifies the corresponding elements of its underlying array names
    fmt.Println(a, b)
    fmt.Println(names)
}

Output:

[Ram Lakshman Parshuram Samvad]
[Ram Lakshman] [Lakshman Parshuram]
[Ram Krishna] [Krishna Parshuram]
[Ram Krishna Parshuram Samvad]

Slice literals

A slice literal is like an array literal without the length.

This is an array literal:
[3]bool{true, true, false}

And this creates the same array as above, then builds a slice that references it:
[]bool{true, true, false}

package main

import "fmt"

func main() {
    q := []int{2, 3, 5, 7, 11, 13}
    fmt.Println(q)

    r := []bool{true, false, true, true, false, true}
    fmt.Println(r)

    s := []struct {
        i int
        b bool
    }{
        {2, true},
        {3, false},
        {5, true},
        {7, true},
        {11, false},
        {13, true},
    }
    fmt.Println(s)
}

Output:

[2 3 5 7 11 13]
[true false true true false true]
[{2 true} {3 false} {5 true} {7 true} {11 false} {13 true}]

Slice Defaults

When slicing, you may omit the high or low bounds to use their defaults instead.
The default is zero for the low bound and the length of the slice for the high bound.

For the array

var a [10]int

these slice expressions are equivalent:

a[0:10]
a[:10]
a[0:]
a[:]

package main

import "fmt"

func main() {
    s := []int{2, 3, 5, 7, 11, 13}

    s = s[1:4] // [3 5 7]
    fmt.Println(s)

    s = s[:2] // [3 5]
    fmt.Println(s)

    s = s[1:] // [5]
    fmt.Println(s)
}

Output:

[3 5 7]
[3 5]
[5]

Slice length & capacity

A slice has both a length and a capacity.
The length of a slice is the number of elements it contains.
The capacity of a slice is the number of elements in the underlying array, counting from the first element in the slice.
The length and capacity of a slice s can be obtained using the expressions len(s) and cap(s).

package main

import "fmt"

func main() {
    s := []int{2, 3, 5, 7, 11, 13}
    printSlice(s)

    // Slice the slice to give it zero length.
    s = s[:0]
    printSlice(s)

    // Extend its length.
    s = s[:4]
    printSlice(s)

    // Drop its first two values.
    s = s[2:]
    printSlice(s)
}

func printSlice(s []int) {
    fmt.Printf("len=%d cap=%d %v\n", len(s), cap(s), s)
}

Output:

len=6 cap=6 [2 3 5 7 11 13]
len=0 cap=6 []
len=4 cap=6 [2 3 5 7]
len=2 cap=4 [5 7]

Nil Slices

The zero value of a slice is nil.
A nil slice has a length and capacity of 0 and has no underlying array.

package main

import "fmt"

func main() {
    var s []int
    fmt.Println(s, len(s), cap(s))
    if s == nil {
        fmt.Println("nil!")
    }
}

Output:

[] 0 0
nil!

Creating a slice with make

Slices can be created with the built-in make function; this is how you create dynamically-sized arrays.
The make function allocates a zeroed array and returns a slice that refers to that array:

a := make([]int, 5)  // len(a)=5

To specify a capacity, pass a third argument to make:

b := make([]int, 0, 5) // len(b)=0, cap(b)=5

b = b[:cap(b)] // len(b)=5, cap(b)=5
b = b[1:]      // len(b)=4, cap(b)=4

Here's an example:

package main

import "fmt"

func main() {
    a := make([]int, 5)
    printSlice("a", a)

    b := make([]int, 0, 5)
    printSlice("b", b)

    c := b[:2]
    printSlice("c", c)

    d := c[2:5]
    printSlice("d", d)
}

func printSlice(s string, x []int) {
    fmt.Printf("%s len=%d cap=%d %v\n",
        s, len(x), cap(x), x)
}

Output:

a len=5 cap=5 [0 0 0 0 0]
b len=0 cap=5 []
c len=2 cap=5 [0 0]
d len=3 cap=3 [0 0 0]

Slices of Slices

Slices can contain any type, including other slices.

package main

import (
    "fmt"
    "strings"
)

func main() {
    // Create a tic-tac-toe board.
    board := [][]string{
        []string{"_", "_", "_"},
        []string{"_", "_", "_"},
        []string{"_", "_", "_"},
    }

    // The players take turns.
    board[0][0] = "X"
    board[2][2] = "O"
    board[1][2] = "X"
    board[1][0] = "O"
    board[0][2] = "X"

    for i := 0; i < len(board); i++ {
        fmt.Printf("%s\n", strings.Join(board[i], " "))
    }
}

Output:

X _ X
O _ X
_ _ O

Appending to a Slice

To append elements to a slice in Go, you can use the built-in append() function. The append() function takes a slice and one or more elements as arguments, and it returns a new slice with the appended elements. Here's an example:

package main

import "fmt"

func main() {
    var s []string
    printSlice(s)

    // append works on nil slices.
    s = append(s, "Ram")
    printSlice(s)

    // The slice grows as needed.
    s = append(s, "Lakshman")
    printSlice(s)

    // We can add more than one element at a time.
    s = append(s, "Parshuram", "Samvad", "Krishna", "Balarama")
    printSlice(s)
}

func printSlice(s []string) {
    fmt.Printf("len=%d cap=%d %v\n", len(s), cap(s), s)
}

Output:

len=0 cap=0 []
len=1 cap=1 [Ram]
len=2 cap=2 [Ram Lakshman]
len=6 cap=6 [Ram Lakshman Parshuram Samvad Krishna Balarama]

The range form of the for loop iterates over a slice or map.
When ranging over a slice, two values are returned for each iteration. The first is the index, and the second is a copy of the element at that index.
You can skip the index or value by assigning to _.

for i, _ := range pow
for _, value := range pow

If you only want the index, you can omit the second variable.

for i := range pow

Here's an example:

package main

import "fmt"

var pow = []int{1, 2, 4, 8, 16, 32, 64, 128}

func main() {
    for i, v := range pow {
        fmt.Printf("2**%d = %d\n", i, v)
    }

    fmt.Println()

    pow := make([]int, 10)
    for i := range pow {
        pow[i] = 1 << uint(i) // == 2**i
    }
    for _, value := range pow {
        fmt.Printf("%d ", value)
    }
}

Output:

2**0 = 1
2**1 = 2
2**2 = 4
2**3 = 8
2**4 = 16
2**5 = 32
2**6 = 64
2**7 = 128

1 2 4 8 16 32 64 128 256 512

Maps

In Go, a map is a built-in data structure that represents an unordered collection of key-value pairs. It is similar to dictionaries or hash tables in other programming languages. Maps provide an efficient way to store and retrieve values based on unique keys. Here's an example of using maps in Go:

package main

import "fmt"

type Vertex struct {
    Lat, Long float64
}

var m = map[string]Vertex{
    "Bell Labs": Vertex{
        40.68433, -74.39967,
    },
    "Google": Vertex{
        37.42202, -122.08408,
    },
}

func main() {
    fmt.Println(m["Bell Labs"])
    fmt.Println(m)  
}

Output:

{40.68433 -74.39967}
map[Bell Labs:{40.68433 -74.39967} Google:{37.42202 -122.08408}]

Mutating Map

Insert or update an element in map m:

m[key] = elem

Retrieve an element:

elem = m[key]

Delete an element:

delete(m, key)

Test that a key is present with a two-value assignment:

elem, ok = m[key]

If key is in m, ok is true. If not, ok is false.
If key is not in the map, then elem is the zero value for the map's element type.

package main

import "fmt"

func main() {
    m := make(map[string]int)

    m["Answer"] = 177
    fmt.Println("The value:", m["Answer"])

    m["Answer"] = 146
    fmt.Println("The value:", m["Answer"])

    delete(m, "Answer")
    fmt.Println("The value:", m["Answer"])

    v, ok := m["Answer"]
    fmt.Println("The value:", v, "Present?", ok)
}

Output:

The value: 177
The value: 146
The value: 0
The value: 0 Present? false

Function Values

Functions are values too. They can be passed around just like other values.
Function values may be used as function arguments and return values.

package main

import (
    "fmt"
    "math"
)

func compute(fn func(float64, float64) float64) float64 {
    return fn(3, 4)
}

func main() {
    hypot := func(x, y float64) float64 {
        return math.Sqrt(x*x + y*y)
    }
    fmt.Println(hypot(5, 12)) // sqrt(5^2 + 12^2) = sqrt(144 + 25) = 13

    fmt.Println(compute(hypot)) // sqrt(3^2 + 12^2) = sqrt(16+9) = 5

    fmt.Println(compute(math.Pow)) // pow(3,4) = 3^4 = 81
}

Output:

13
5
81

Thank you for diving into this chapter of the blog! We've covered a lot of ground, but the journey doesn't end here. The next chapter awaits, ready to take you further into the depths of our topic.

To continue reading and explore the next chapter, simply follow this link: Link To Next Chapter

"Unlock the enigmatic world of pointers, structs, arrays, slices, and maps – where syntax humorously baffles and amuses!✨👀"

Demystifying Ethereum: Unraveling the World of Decentralized Finance and Smart Contracts

Sahil Sojitra — Fri, 26 May 2023 07:36:25 +0000

What is Ethereum?

Ethereum is a decentralized platform that allows people to build and run smart contracts and decentralized applications (DApps). It was created to provide a way for developers to create and execute programs on a global network of computers called the Ethereum Virtual Machine (EVM).

At its core, Ethereum is based on blockchain technology, just like Bitcoin. But while Bitcoin focuses mainly on being a digital currency, Ethereum expands on that idea by enabling the development of a wide range of applications beyond money.

The main concept in Ethereum is the smart contract. A smart contract is like a digital agreement that automatically executes when certain conditions are met. It's called "smart" because it can self-execute without the need for intermediaries like banks or lawyers. Smart contracts are written in a programming language called Solidity.

Ethereum's blockchain is a decentralized ledger that records all transactions and smart contracts. It's maintained by a network of computers, or nodes, around the world. These nodes work together to validate and store the data, ensuring that it remains secure and tamper-proof.

The Ethereum platform has its own cryptocurrency called Ether (ETH). Ether is used to power the network and incentivize the nodes to perform computations and validate transactions. It's also used to pay for gas, which is the computational power required to execute operations on the Ethereum network. Gas helps prevent spam and abuse on the network.

One of the significant advantages of Ethereum is its ability to support decentralized applications. DApps are like traditional apps, but instead of relying on a centralized server, they run on the Ethereum blockchain. This means they are resistant to censorship and single points of failure. DApps can be anything from decentralized finance (DeFi) platforms and games to supply chain management systems and prediction markets.

Ethereum is constantly evolving, and upgrades like Ethereum 2.0 aim to address scalability and improve the network's efficiency. The upgrade will transition Ethereum from a proof-of-work (PoW) consensus mechanism to a proof-of-stake (PoS) mechanism, which is expected to make the network faster and more energy-efficient.

Compared to Bitcoin

Many people entering the world of cryptocurrencies often start with Bitcoin. However, Ethereum stands out with its distinctive features and purpose. we'll explore the ways in which Ethereum differs from Bitcoin and why it has gained significant attention and adoption in the blockchain space.

1. Purpose: Beyond Digital Currency:
While Bitcoin is primarily known as a decentralized digital currency, Ethereum goes beyond that. Ethereum serves as a platform for decentralized applications (DApps) and smart contracts. It acts as a world computer, allowing developers to create and execute complex applications and agreements directly on the blockchain.

2. Construction: A Programmable Blockchain:
Bitcoin's scripting language is limited, mainly focusing on simple spending conditions. In contrast, Ethereum is built as a general-purpose programmable blockchain. It incorporates a virtual machine capable of executing code of any complexity, making it Turing complete. This means Ethereum can function as a versatile and powerful computer network.

3. Smart Contracts: Enabling Automation and Agreements:
One of Ethereum's key differentiators is the concept of smart contracts. These self-executing agreements automatically execute when specific conditions are met. With Ethereum's programming capabilities, developers can create sophisticated smart contracts that automate various processes, from financial transactions to supply chain management.

4. Ether as a Utility Currency:
While Bitcoin serves as a digital currency and a store of value, Ethereum's native currency, called ether (ETH), plays a different role. Ether acts as a utility currency within the Ethereum ecosystem, used to pay for computational resources and transactions on the platform. It fuels the execution of smart contracts and incentivizes participants to secure the network.

5. Development Community and Ecosystem:
Bitcoin has a strong focus on financial use cases, while Ethereum attracts a broader range of developers and projects due to its programmability. Ethereum has become a hub for decentralized finance (DeFi), gaming, non-fungible tokens (NFTs), and more. Its thriving ecosystem offers a multitude of decentralized applications and innovative projects.

Components of Blockchain

The components of an open, public blockchain are (usually):

1. Peer-to-Peer Network: Connecting Participants:
An open blockchain relies on a peer-to-peer (P2P) network, enabling direct communication and interaction between participants. This decentralized network propagates transactions and blocks of verified transactions across the network, ensuring that all participants have access to the latest information.

2. Transactions: Representing State Transitions:
Transactions are the messages within the blockchain network that represent state transitions. These transactions encapsulate information such as the sender, recipient, and any associated data. They play a crucial role in updating the state of the blockchain and initiating changes in the system.

3. Consensus Rules: Governing Validity:
Consensus rules define what constitutes a valid transaction and determine the conditions for a valid state transition. These rules ensure that all participants in the blockchain network agree on the correctness and validity of transactions and prevent malicious activities.

4. State Machine: Processing Transactions:
The state machine is a fundamental component that processes transactions based on the consensus rules. It executes the logic encoded within smart contracts or protocols to update the state of the blockchain. This processing step is crucial in maintaining the integrity and accuracy of the blockchain.

5. Cryptographically Secured Blocks: The Journal of State Transitions:
Blocks serve as a journal of all verified and accepted state transitions in the blockchain. Each block contains a set of transactions, along with additional information such as a timestamp and a reference to the previous block. These blocks are cryptographically secured, ensuring immutability and tamper resistance.

6. Consensus Algorithm: Decentralized Control:
Consensus algorithms are designed to decentralize control over the blockchain network. They require participants to collaborate and agree on the validity of transactions and state transitions. Through mechanisms like proof-of-work or proof-of-stake, consensus algorithms enable the network to reach a collective agreement without relying on a centralized authority.

7. Incentivization Scheme: Economically Securing the State Machine:
To secure the state machine and incentivize participants, open blockchains often employ game-theoretically sound schemes. For example, proof-of-work involves participants competing to solve complex mathematical puzzles, with rewards given to successful miners. This incentivization scheme ensures the economic stability and security of the blockchain network.

8. Open Source Software Implementations:
Open blockchains often have one or more open source software implementations, known as "clients." These clients provide the necessary tools for participants to interact with the blockchain network, including wallet software, node software, and development frameworks.

Most blockchains combine multiple components into a single software client. For example, Bitcoin Core is the reference implementation for Bitcoin, while Ethereum has a reference specification called the Yellow Paper, with multiple clients built according to this specification.

The components of a blockchain include the peer-to-peer network, transactions, consensus rules, state machine, cryptographically secured blocks, consensus algorithm, incentivization scheme, and open source software implementations. These components work together to create a decentralized and transparent system.

As the blockchain landscape has evolved, it has become essential to use qualifiers to understand the characteristics of a particular blockchain system. Qualifiers such as open, public, global, decentralized, neutral, and censorship-resistant help identify the important emergent characteristics of a blockchain.

Today, there is a wide variety of blockchains, each with different properties and characteristics. It is crucial to ask questions and seek clarification when someone mentions a "blockchain." Asking for a description of the components and inquiring whether the blockchain exhibits qualities like being open, public, etc., helps to understand its nature.

Birth of Ethereum

Ethereum emerged as a solution to extend the capabilities of Bitcoin and address the need for more flexibility and freedom in blockchain applications. we'll explore the journey of Ethereum's conception, the challenges faced by developers, and how Ethereum's founders pioneered a general-purpose blockchain to revolutionize decentralized applications.

1. The Need for Innovation:
Recognizing the potential beyond cryptocurrency, developers sought to go beyond the limitations of the Bitcoin model. However, building on Bitcoin required workarounds due to its intentional constraints. The constraints on transaction types, data types, and data storage hindered the development of diverse applications directly on the Bitcoin blockchain.

2. Vitalik Buterin's Proposal:
In 2013, Vitalik Buterin, a young programmer and Bitcoin enthusiast, began envisioning a more versatile blockchain. He proposed the idea of a Turing-complete, general-purpose blockchain to the Mastercoin team. While the proposal was too radical for their development roadmap, it gained attention and feedback from a small group of individuals.

3. Collaboration and Refinement:
Vitalik shared an early draft of the Ethereum whitepaper, receiving input and suggestions from a few dozen people. Notable contributors included Andreas M. Antonopoulos, who showed interest in Ethereum's potential but focused on writing his book at the time. Dr. Gavin Wood, a skilled C++ programmer, joined forces with Vitalik and became Ethereum's co-founder, codesigner, and CTO.

4. The Birth of Ethereum:
From December 2013 onward, Vitalik and Gavin collaborated to refine and evolve the Ethereum concept. They constructed the protocol layer that would become Ethereum, envisioning a blockchain that could support a wide range of applications through programmability. Their goal was to abstract the complexities of peer-to-peer networks, blockchains, and consensus algorithms, providing a secure and deterministic environment for decentralized applications.

5. Combining Innovations:
Similar to Satoshi Nakamoto, the creator of Bitcoin, Vitalik and Gavin combined existing technologies with new inventions to demonstrate their vision. They meticulously developed and refined the Ethereum platform over the years, culminating in the mining of the first Ethereum block on July 30, 2015.

Ethereum's Four Stages Development

Ethereum’s development was planned over four distinct stages, with major changes occurring at each stage. A stage may include subreleases, known as "hard forks," that change functionality in a way that is not backward compatible.

The four main development stages are codenamed Frontier, Homestead, Metropolis, and Serenity. The intermediate hard forks that have occurred to date are codenamed Ice Age, DAO, Tangerine Whistle, Spurious Dragon, Byzantium, Constantinople/St. Petersburg, Istanbul and Muir Glacier. Both the development stages and the intermediate hard forks are shown on the following timeline, which is "dated" by block number:

1. Frontier:
Frontier was the initial stage of Ethereum, commencing on July 30, 2015, and lasting until March 2016. It marked the launch of the Ethereum blockchain, allowing developers and users to engage with the network and experiment with decentralized applications.

2. Homestead:
Homestead, launched in March 2016, represented the second stage of Ethereum. It introduced improvements to enhance the stability and security of the platform, solidifying Ethereum as a reliable ecosystem for decentralized applications.

3. Metropolis:
Metropolis, the third stage of Ethereum, was divided into two hard forks: Byzantium and Constantinople/St. Petersburg. Byzantium, implemented in October 2017 at block #4,370,000, introduced low-level functionalities and adjusted the block reward and difficulty. Constantinople/St. Petersburg, initially planned as the second part of Metropolis, underwent a last-minute postponement due to a critical bug discovery and was eventually renamed St. Petersburg.

4. Serenity:
Serenity represents the final stage of Ethereum's development, often referred to as "Ethereum 2.0." It involves a comprehensive overhaul of Ethereum's infrastructure to address scalability, security, and sustainability. Serenity will introduce significant changes, including the transition from proof-of-work (PoW) to proof-of-stake (PoS) consensus, shard chains for improved scalability, and other enhancements.

Ethereum has successfully completed the Berlin and London hard forks, marking significant milestones in its development. These upgrades have brought notable improvements to the Ethereum network, paving the way for the final stage of development known as Serenity or Ethereum 2.0.

Berlin: Advancing Ethereum's Capabilities
The Berlin hard fork, already implemented, brought important enhancements to Ethereum. It focused on optimizing the platform's functionality by introducing several Ethereum Improvement Proposals (EIPs). These EIPs aimed to improve the efficiency of smart contract execution, enhance transaction processing, and streamline overall network performance. With Berlin, Ethereum took a significant step forward in preparing the foundation for Ethereum 2.0.
London: Revolutionizing Transaction Fees
The London hard fork is the next milestone on Ethereum's roadmap. It introduces a groundbreaking change to the transaction fee structure through the implementation of EIP-1559. This upgrade aims to improve the user experience by introducing a more predictable fee mechanism and reducing transaction costs. Additionally, EIP-3554 will adjust the mining reward structure, further enhancing the network's sustainability. London is expected to be deployed in the coming months, bringing important improvements to Ethereum's fee model.

Ethereum's Components

In Ethereum, the key components of a blockchain system are:

1. P2P network: Ethereum operates on a network called the Ethereum main network, which enables peer-to-peer communication among participants.

2. Consensus rules: The rules that govern how Ethereum operates are defined in the Yellow Paper, a reference specification for the platform.

3. Transactions: Ethereum transactions are messages exchanged on the network and include information such as the sender, recipient, value, and data payload.

4. State machine: Ethereum uses the Ethereum Virtual Machine (EVM), a virtual machine that executes smart contracts written in high-level languages like Solidity. These contracts are compiled into bytecode for execution on the EVM.

5. Data structures: Ethereum's state is stored in a database, typically LevelDB, on each node. The data is organized using a Merkle Patricia Tree, a hashed data structure.

6. Consensus algorithm: Currently, Ethereum uses Nakamoto Consensus, the same model as Bitcoin, where blocks are created through proof-of-work (PoW). However, there are plans to transition to a proof-of-stake (PoS) system called Casper in the future.

7. Economic security: Ethereum currently relies on the Ethash PoW algorithm for security. However, as the platform moves towards PoS, this algorithm will be replaced.

8. Clients: Ethereum has multiple client software implementations, including Go-Ethereum (Geth) and Parity, which enable interaction with the Ethereum network.

These components work together to power Ethereum's decentralized and programmable blockchain platform, facilitating the execution of smart contracts and the secure transfer of digital assets.

Ethereum Basics

Ethereum’s currency unit is called ether, identified also as "ETH" or with the symbols Ξ (from the Greek letter "Xi" that looks like a stylized capital E) or, less often, ♦: for example, 1 ether, or 1 ETH, or Ξ1, or ♦1.

Ether is subdivided into smaller units, down to the smallest unit possible, which is named wei. One ether is 1 quintillion wei (1 * 1018 or 1,000,000,000,000,000,000). You may hear people refer to the currency "Ethereum" too, but this is a common beginner’s mistake. Ethereum is the system, ether is the currency.

The value of ether is always represented internally in Ethereum as an unsigned integer value denominated in wei. When you transact 1 ether, the transaction encodes 1000000000000000000 wei as the value.

Choosing an Ethereum Wallet

The term "wallet" has come to mean many things, although they are all related and on a day-to-day basis boil down to pretty much the same thing. We will use the term "wallet" to mean a software application that helps you manage your Ethereum account. In short, an Ethereum wallet is your gateway to the Ethereum system. It holds your keys and can create and broadcast transactions on your behalf. Choosing an Ethereum wallet can be difficult because there are many different options with different features and designs. Some are more suitable for beginners and some are more suitable for experts. The Ethereum platform itself is still being improved, and the "best" wallets are often the ones that adapt to the changes that come with the platform upgrades.

But don’t worry! If you choose a wallet and don’t like how it works—or if you like it at first but later want to try something else—you can change wallets quite easily. All you have to do is make a transaction that sends your funds from the old wallet to the new wallet, or export your private keys and import them into the new one.

The following are some good starter wallets:

1. MetaMask
MetaMask is a browser extension wallet that runs in your browser (Chrome, Firefox, Opera, or Brave Browser). It is easy to use and convenient for testing, as it is able to connect to a variety of Ethereum nodes and test blockchains. MetaMask is a web-based wallet that also includes mobile apps for both iOS and Android.

2. Jaxx
Jaxx is a multiplatform and multicurrency wallet that runs on a variety of operating systems, including Android, iOS, Windows, macOS, and Linux. It is often a good choice for new users as it is designed for simplicity and ease of use. Jaxx is either a mobile or a desktop wallet, depending on where you install it.

3. MyEtherWallet (MEW)
MyEtherWallet is primarily a web-based wallet that runs in any browser. It is also available on Android and iOS. It has multiple sophisticated features we will explore in many of our examples.

4. Emerald Wallet
Emerald Wallet is designed to work with the Ethereum Classic blockchain, but is compatible with other Ethereum-based blockchains. It’s an open source desktop application and works under Windows, macOS, and Linux. Emerald Wallet can run a full node or connect to a public remote node, working in a "light" mode. It also has a companion tool to do all operations from the command line.

Why Learn Ethereum?

Blockchains have a very steep learning curve, as they combine multiple disciplines into one domain: programming, information security, cryptography, economics, distributed systems, peer-to-peer networks, etc. Ethereum makes this learning curve a lot less steep, so you can get started quickly. But just below the surface of a deceptively simple environment lies a lot more. As you learn and start looking deeper, there’s always another layer of complexity and wonder.

Ethereum is a great platform for learning about blockchains and it’s building a massive community of developers, faster than any other blockchain platform. More than any other, Ethereum is a developer’s blockchain, built by developers for developers. A developer familiar with JavaScript applications can drop into Ethereum and start producing working code very quickly. For the first few years of Ethereum’s life, it was common to see T-shirts announcing that you can create a token in just five lines of code. Of course, this is a double-edged sword. It’s easy to write code, but it’s very hard to write good and secure code.

"Jump into Ethereum's world of infinite possibilities, where understanding smart contracts is as easy as explaining quantum mechanics to a goldfish✨😅"