DEV Community: Nick Mudge

Understanding EIP-2535 Diamonds: The Future of Smart Contract Design

Nick Mudge — Wed, 22 May 2024 17:42:32 +0000

In the ever-evolving world of blockchain technology, Ethereum Improvement Proposals (EIPs) play a crucial role in advancing the capabilities and functionalities of the Ethereum network. One such significant proposal is EIP-2535 Diamonds. This proposal introduces a standard for organizing and managing complex smart contracts, offering unprecedented flexibility and scalability. In this blog post, we'll explore what EIP-2535 Diamonds are, their benefits, and how you can use them in your smart contract development.

What is EIP-2535?

EIP-2535, proposed by Nick Mudge in 2020, introduces the concept of "Diamonds," a modular and scalable architecture for smart contracts on the Ethereum blockchain. Unlike traditional smart contracts, which can become cumbersome and difficult to manage as they grow in complexity, Diamonds allow developers to break down their contracts into smaller, more manageable pieces called "facets."

Key Components of EIP-2535

Diamond Contract: This is the core contract that acts as a hub, managing the various facets. It delegates function calls to the appropriate facet based on the function signature.
Facets: These are modular contracts that contain specific functionalities. A single Diamond can have multiple facets, each responsible for different parts of the overall logic.
Selectors: Function selectors are used to route calls to the correct facet. The Diamond contract maintains a lookup table of function selectors to the corresponding facet addresses.
Loupe Functions: These are introspection functions that allow developers to query the Diamond for information about its facets and the functions they provide.

Benefits of EIP-2535

Modularity: By breaking down a smart contract into smaller facets, developers can manage and update specific parts of the contract without affecting the entire system.
Scalability: Diamonds enable the development of large and complex systems that can grow organically. New functionalities can be added by deploying new facets.
Upgradeability: Individual facets can be upgraded independently, allowing for flexible and seamless updates without redeploying the entire contract.
Reduced Gas Costs: Since only the facets that are updated need to be redeployed, the overall gas costs for contract upgrades can be significantly lower.
Organization: By organizing code into facets, developers can keep their codebase clean and easier to navigate, enhancing maintainability.

How to Use EIP-2535 Diamonds

Using EIP-2535 Diamonds involves several steps, from setting up your development environment to deploying and interacting with your Diamond contract. Here's a step-by-step guide:

1. Setting Up Your Environment

First, ensure you have a development environment set up with tools like Hardhat or Truffle, and a coding editor like VSCode.

npm install --save-dev hardhat

2. Create Your Diamond Contract

Create the Diamond contract that will act as the central hub. This contract includes the logic to delegate function calls to the appropriate facets.

// Diamond.sol
pragma solidity ^0.8.0;

import "./IDiamondCut.sol";

contract Diamond {
    // Diamond storage
    struct Facet {
        address facetAddress;
        bytes4[] selectors;
    }

    mapping(bytes4 => address) public selectorToFacet;

    constructor(IDiamondCut.FacetCut[] memory _diamondCut) {
        // Implement diamond cut
    }

    fallback() external payable {
        address facet = selectorToFacet[msg.sig];
        require(facet != address(0), "Function does not exist");
        assembly {
            calldatacopy(0, 0, calldatasize())
            let result := delegatecall(gas(), facet, 0, calldatasize(), 0, 0)
            returndatacopy(0, 0, returndatasize())
            switch result
            case 0 {revert(0, returndatasize())}
            default {return(0, returndatasize())}
        }
    }
}

3. Define Your Facets

Create facet contracts that contain specific functionality. For example, you might have one facet for token management and another for governance.

// TokenFacet.sol
pragma solidity ^0.8.0;

contract TokenFacet {
    mapping(address => uint256) balances;

    function transfer(address to, uint256 amount) external {
        require(balances[msg.sender] >= amount, "Insufficient balance");
        balances[msg.sender] -= amount;
        balances[to] += amount;
    }

    function balanceOf(address account) external view returns (uint256) {
        return balances[account];
    }
}

4. Deploy Your Diamond and Facets

Deploy your Diamond contract and then the facet contracts. After deploying, you need to link the facets to the Diamond using the Diamond Cut mechanism.

async function main() {
    const Diamond = await ethers.getContractFactory("Diamond");
    const diamond = await Diamond.deploy();
    await diamond.deployed();
    console.log("Diamond deployed to:", diamond.address);

    const TokenFacet = await ethers.getContractFactory("TokenFacet");
    const tokenFacet = await TokenFacet.deploy();
    await tokenFacet.deployed();
    console.log("TokenFacet deployed to:", tokenFacet.address);

    // Add facets to the Diamond
    await diamond.diamondCut([
        {
            facetAddress: tokenFacet.address,
            action: 0, // Add
            functionSelectors: getSelectors(tokenFacet)
        }
    ], ethers.constants.AddressZero, "0x");
}

5. Interact with Your Diamond

Once deployed, you can interact with your Diamond contract. Calls to the Diamond will be routed to the appropriate facet.

const diamond = await ethers.getContractAt("Diamond", diamondAddress);
const tokenFacet = await ethers.getContractAt("TokenFacet", diamondAddress);

await tokenFacet.transfer(receiverAddress, amount);
const balance = await tokenFacet.balanceOf(receiverAddress);
console.log("Balance:", balance.toString());

A fully working and complete implementation of EIP-2535 Diamonds is here: https://github.com/mudgen/diamond-1-hardhat.

To date more than 100 projects have used EIP-2535 Diamonds. A list is available here: https://github.com/mudgen/awesome-diamonds?tab=readme-ov-file#projects-using-diamonds

Conclusion

EIP-2535 Diamonds provide a powerful framework for building modular, scalable, and upgradeable smart contracts on Ethereum. By breaking down contracts into facets, developers can manage complex systems more effectively, reduce gas costs, and ensure seamless upgrades. Whether you are building decentralized finance (DeFi) applications, gaming platforms, or any other type of dApp, Diamonds offer a robust solution to the challenges of smart contract development. Start exploring EIP-2535 Diamonds today and take your blockchain projects to the next level!

How to solve OpenSea's "We couldn’t find this contract." error for ERC721 Validation

Nick Mudge — Tue, 01 Mar 2022 16:22:15 +0000

You may have a perfectly valid ERC721 contract yet OpenSea gives you an error when you try to add your ERC721 contract to OpenSea.

This is the URL for adding an ERC721 collection to OpenSea's testnet which uses Rinkeby: https://testnets.opensea.io/get-listed/step-two

This is the error you can get:

We couldn't find this contract. Please ensure that this is a valid ERC721 or ERC1155 contract deployed on Rinkeby and that you have already minted items on the contract.

You may get this error even if your contract is compliant with the ERC721 standard. A team from the Opinyour project and myself debugged OpenSea's ERC721 validation and discovered what we think is the problem and a solution that works.

How OpenSea ERC721 Validation Works

From behavior of OpenSea's ERC721 validation we discovered how OpenSea validates contracts as ERC721 or not. This investigation and discovery occurred on the Rinkeby network. Based on observed behavior this is how OpenSea ERC721 validation works:

OpenSea automatically monitors all new contracts that are deployed. The moment OpenSea detects a new contract has been deployed it calls contract.supportsInterface(type(IERC721).interfaceId) on that contract. If the function call returns true then OpenSea adds it to its list or database of potential ERC721 contracts. If the function call returns false then OpenSea will not detect it as an ERC721 contract and any attempts at validation fails.
OpenSea checks during deployment and after deployment (for any contract that passes step 1 above) if any ERC721 Transfer events have been emitted from a contract. If not then any validation fails.

If a contract passes step 1 and 2 then it will pass ERC721 validation on OpenSea's validation webpage.

When OpenSea ERC721 Validation Fails

For a lot of contracts OpenSea's current ERC721 validation strategy works. But there are two cases that I know where OpenSea will fail to validate smart contracts that comply fully with the ERC721 standard. Here are the two cases:

If OpenSea's infrastructure that processes newly deployed contracts on a blockchain stops working or develops a backlog then newly deployed ERC721 contracts will fail validation until OpenSea's monitoring infrastructure is working again or until the backlog of unprocessed contracts is handled.
A proxy contract that is upgraded after deployment to support ERC721 can fail validation. The reason is because OpenSea validates contracts when they are deployed, but such contracts don't implement ERC721 until after they are deployed. See solutions to this below.

Here is an example of a scenario where ERC721 validation fails and how to get it working:

We deploy an EIP2535 diamond.
Right after deployment we upgrade the diamond to fully implement the ERC721 standard.

Note: Introduction article to EIP2535 Diamonds here: https://eip2535diamonds.substack.com/p/introduction-to-the-diamond-standard

As mentioned before such a diamond contract can fail OpenSea ERC721 validation if OpenSea validates the diamond before the upgrade transaction adds the ERC721 functions. Also understand that it is possible for ERC721 validation to pass in this case if OpenSea has a blacklog of contracts to process and by the time it processes our diamond the upgrade transaction has been added to the blockchain.

The very specific solution to this is to add the supportsInterface(bytes4 _interfaceId) function to the diamond when it is deployed. This is easily done by adding the function to the diamond in the constructor function of the diamond. It could also be added directly to the diamond as an immutable function. Also be sure that the function returns true for the type(IERC721).interfaceId argument.

Here is a link to code that shows an example of adding the supportsInterface function to a diamond in its constructor function: https://gist.github.com/mudgen/fe2ee945446e64c56c9c11c1796c1f3b

Because we now understand why validation fails and generally how to fix it (add supportsInterface function during deployment), there are a number of different ways to implement a solution.

One solution is to add all the ERC721 functions to the diamond during deployment. It is possible to add the ERC721 functions to the diamond in the diamond's proxy constructor function.

Solidity: AppStorage for distinguishing state variables and preventing name clashes

Nick Mudge — Thu, 13 Jan 2022 21:47:51 +0000

Solidity state variables, immutable variables, constants, local variables and function names can look the same and it is possible to have name clashes, where for example a local variable or function name is named the same thing as a state variable, which can reduce code clarity and cause frustration.

AppStorage is a variable naming technique which makes state variables clear in your code and prevents name clashes with other kinds of variables and function names.

This is how it is done: Define a struct in a file and name the struct AppStorage. Define all the state variables in your application within the AppStorage struct.

Then in your smart contracts import the AppStorage struct. Then define this state variable: AppStorage internal s;.

That's it. Now you can access all your state variables by prepending an "s." to them.

Here is a simple example:

Define AppStorage that contains all your state variables:

// AppStorage.sol
struct AppStorage {
  uint256 secondVar;
  uint256 firstVar;
  uint256 lastVar;
  ...
}

Import it and use it in contracts:

import "./AppStorage.sol"

contract Staking {
  AppStorage internal s;

  function myFacetFunction() external {
    s.lastVar = s.firstVar + s.secondVar;
  }

How Diamond Upgrades Work

Nick Mudge — Fri, 14 May 2021 04:20:19 +0000

To understand diamond upgrades, let's first understand how function calls work in contracts created by Solidity.

How Function Calls Work

In order for a transaction to execute a function on a smart contract there needs to be a way for a transaction to specify to the contract what function to execute. This is done by a transaction supplying a four-byte value that identifies for a contract what function to execute.

The four-byte value is called a function selector. It consists of the first four bytes of a hash of a string of a function signature.

A function signature is a string that consists of a function name and the types of its arguments. Let's look at an example. Here's a function signature: "transfer(address,uint256)"

Here we get the first four bytes of a hash of that function signature:

bytes functionSelector = bytes4(keccak256("transfer(address,uint256)"));

The function selector is0xa9059cbb. So if the first four bytes of transaction data is that value and the transaction is sent to a contract, then it tells that contract to execute the "transfer(address,uint256)" function.

How Diamond Upgrades Work

Diamonds store function selectors and contract addresses of contracts that have the functions that the function selectors identify. The contracts that diamonds use are called facets.

A diamond has a mapping that maps function selectors to facet addresses.

Adding functions to a diamond means adding function selectors to a diamond and adding the facet addresses of the facets that have the functions that the function selectors identify.

Replacing functions means associating existing function selectors in a diamond with different facet addresses.

Removing functions means removing function selectors and their facet addresses from the diamond.

The diamondCut function is used to upgrade diamonds. It has a _diamondCut argument that consists of an array of function selectors and facet addresses and an action which specifies whether to add, replace or remove them. Here's an example of a value for the _diamondCut argument:

[
  {
    facetAddress: '0x0DCd1Bf9A1b36cE34237eEaFef220932846BCD82',
    action: 0,
    functionSelectors: [
      '0x4cd5d1f7',
      '0x24a6665e',
      '0x4113e5ca',
      '0x49aa1f27',
      '0x24d86f00',
      '0x2e0bcb43',
      '0xbec10cde',
      '0xa3ea00f1'
    ]
  },
  {
    facetAddress: '0x549549085493058324890438543949485a499A1b',                  
    action: 1,
    functionSelectors: [
      '0x45943954',
      '0xa4349495',
      '0x9684834e'
    ]
  }  
]

In the example above the action value of 0 means adding new functions. The action value of 1 means replacing the functions by replacing the facet address for them.

In Javascript scripts the FacetCutAction object can used to more explicitly show the action. Here is its definition:

const FacetCutAction = { Add: 0, Replace: 1, Remove: 2 }

Here is an example of a Javascript script that creates the _diamondCut argument:

const FacetCutAction = { Add: 0, Replace: 1, Remove: 2 }

  const newFuncs = [
    getSelector('function setSleeves(tuple(uint256 sleeveId, uint256 wearableId)[] calldata _sleeves) external'),
    getSelector('function updateSvg(string calldata _svg, tuple(bytes32 svgType, uint256[] ids, uint256[] sizes)[] calldata _typesAndIdsAndSizes) external')
  ]
  const existingFuncs = getSelectors(facet).filter(selector => !newFuncs.includes(selector))

  const _diamondCut = [
    {
      facetAddress: facet.address,
      action: FacetCutAction.Add,
      functionSelectors: newFuncs
    },
    {
      facetAddress: facet.address,
      action: FacetCutAction.Replace,
      functionSelectors: existingFuncs
    }
  ]
  console.log(_diamondCut)

The AavegotchiDiamond has many upgrade scripts. Check them out for more code examples.

You may want to initialize state after an upgrade. That is what the second and third arguments of the diamondCut function are for. The _init argument holds the contract address of a function to call to initialize the state of the diamond. The _calldata argument holds a function call to send to the contract at _init. The function call is executed with delegatecall so that the diamond's state is affected and can be initialized.

Executing an initialization function by using the _init and _calldata arguments when making an upgrade means that the upgrade and state initialization are done in the same transaction. This is good because it prevents the diamond from getting into a bad or inconsistent state, which could possibly happen if an upgrade and state initialization are done in separate transactions.

How to Debug Code & Debugging Techniques

Nick Mudge — Sun, 02 May 2021 00:26:26 +0000

You might not know that you can solve most of your software or coding problems yourself and it is important that you do so rather than seek help directly from others. So the first thing to become more aware of is that it is possible to solve your own coding problems, and for many coding problems you can solve them easily and quickly. You just need to know how and practice and get better at debugging. It can actually be fun and rewarding.

It is important to know how to debug code. It is a skill that can be developed and improved. The more you debug code the better you get at it. It is like a muscle, the more you flex it and use it the better and stronger and faster you get at it.

Why is being good at debugging important?

The old fish story is applicable here:

Give a man a fish and you feed him for a day; teach a man to fish and you feed him for a lifetime.

If your code doesn't work you could ask a senior developer about it. So the senior developer uses his time to look at, understand and debug your code for you. He finds the problem and gives you the solution. You learn from it. That's good. But a bit later your code stops working and so you ask the senior developer about it and the senior developer looks at it, understands and debugs your code and you learn from it. And the cycle repeats. You do learn this way but it is not a good way to learn. You are reliant on others when you get stuck. You are stuck and stopped and have to wait until someone else can help you. You can't do anything new without someone else to help you. You are using up the valuable time of a senior developer. You might be a developer that was hired to solve problems, not find or create problems for others to solve. It doesn't have to be this way. If you are good at debugging then you can solve your own problems and be self-reliant. You will save tons of time and you won't be stopped when you start working on new things or using new technologies. The good thing is that debugging is a skill that you can learn and get better at.

You might have excuses for not solving problems or successfully debugging things, or maybe others have given you excuses. You will have to give up excuses. Notice you have excuses, if you have them, and let them go. The more you work on your debugging skills and work on debugging things yourself, the better you will get at debugging and solving things.

The benefit of being good at debugging goes beyond getting code to work. Debugging makes you more familiar with your code, other people's code, documentation and tools. Debugging increases your general knowledge and competency and confidence. All these things make you a better developer.

How do you actually debug?

The primary, fundamental thing about code that is not running as expected is that it means you don't know something.

Debugging means finding out what you don't know so you can know it and get your code working right.

Another way to say this is this:

Debugging is increasing your understanding with what you are working with so you make it work right.

Some people make random changes to code in hopes that it works now. That is not debugging. That's a waste of time. That indicates that a person doesn't understand the code in the first place, which is terrible. Nobody should be dealing with code they don't understand. The first step is to understand what you are dealing with. Anything that is done that helps you to understand more is valuable. Experimenting or changing code to see different outputs or results that helps you understand something better is valuable. Always aim for increasing your understanding. When you understand then you can solve with certainty.

I'm going to separate debugging into two separate areas: Lacking Knowledge and Oversight.

Lacking Knowledge

Lacking knowledge means that your code isn't working because you lack knowledge about the tools you are using. There could be code in your script you don't understand, a smart contract you are interacting with that you don't understand or you don't understand something about the execution environment (Hardhat or nodejs) or a library you are using, or configuration or other software running with your script, such as Hardhat Network. Or maybe you don't understand the error message you are getting.

The thing to do is increase your understanding of the code and tools you are using so you can use them correctly.

Any tradesman, musician, mechanic, technical person is expected to know his/her tools. A programmer is no different. Take some time everyday, if you can, to learn more about your programming language, libraries, smart contract standards, smart contracts, programming editor/IDE and other tools. Read documentation and buy and read books about your tools. Watch videos if you like to learn that way.

If your code throws an error then look at every part of the error and if there is an error message then ask yourself if you understand it or not. If you don't understand the error message then your next step is to find out what the error message means.

If you understand what the error message means but still don't understand why it is being thrown then look through your code and see if there is any code you don't understand or might not understand. You should understand all the code in your scripts and if you don't you should take action to understand it.

The best and easiest and most accurate way to learn about functions and features about your programming language and libraries and tools is to read the documentation about them.

Documentation is your best friend. For something you don't understand read the documentation about it. Here are the documentation sources for the tools and software we use:

Javascript: https://developer.mozilla.org/en-US/docs/Web/JavaScript
nodejs: https://nodejs.org/en/docs/
Hardhat: https://hardhat.org/getting-started/
Ethersjs: https://docs.ethers.io/v5/
Solidity: https://docs.soliditylang.org/en/v0.8.4/
Polygon: https://docs.matic.network/docs/develop/getting-started
Aavegotchi: https://wiki.aavegotchi.com/
AavegotchiDiamond: https://docs.aavegotchi.com/

Notice that Stack Overflow and other forum software is not on this list. Usually the documentation from a project is better, more accurate, than other sources so first seek the documentation of a project.

But sometimes what you need to understand or learn is not in the documentation of a project. That's when to use google or other search software and Stack Overflow and forums to find data that you need.

Sometimes the documentation or information you need isn't found searching it on Google or on forums etc. So search the Github issues for the project. It is also possible to look at the source code of the project to understand or get the data you need.

If you still can't find the data you need then there are a number of things you can do. You can create a github issue for the project. You can find the discord server for the project and ask your questions there.

If you still can't find the data you need then you can ask a senior developer. Don't say your code doesn't work and give little data. Ask for data that you need to solve and debug your own code. If you ask for data that is already in the documentation then a senor developer is going to wonder why you are asking him/her about something that is already answered in the documentation. So if you can't find answers in the documentation then it is good to say so. You could say something like this: "I could not find in the documentation [the question], do you know where documentation is for this?" If there isn't documentation for your question then it signals to the senior developer that he/she or someone should write documentation for that thing, which is useful.

Going to a senior developer for things that are already answered in the documentation for a tool is wasting the senior developers time and is wasting the time and effort of the person who wrote the documentation to answer or solve your problems. In addition getting things hand fed from a senior developer doesn't strengthen your debugging muscle and makes you dependent on others. It might save time but it doesn't make you as good as you could be.

Oversight

Sometimes code doesn't work not so much because we don't understand tools but because of some little mistake in code that we missed. Syntax errors and logic errors are examples of this. Sometimes these mistakes are very easy and fast to find and fix and sometimes they take longer to find and fix. The various debugging techniques can be used to find the source of code not working so it can be fixed.

Debugging Techniques

All debugging techniques should help you discover the cause for software not working correctly. It is not enough to just get software to run correctly. You need to understand why it wasn't working correctly. You need to know the cause, source of it working incorrectly. Only then can you be sure that you really fix the problem so it doesn't happen again.

Many debugging techniques help you find where in code a problem is. Once you know where in code a problem is you can focus your attention there to really understand the code there and find the source of a problem.

There are many debugging techniques. These are some of them. Please contact me about adding more if you have some good techniques.

Look at the Error Message

If you get an error then look at the error message. Often somewhere in the stack trace there is an error message that states what the error is. Use that. Also the error may say or indicate where in the code the error is. Looking at and using error messages is one of the most common and first approaches for debugging something.

Program a Little at a Time

When working with unfamiliar code or tools it is important to program a little at a time. Write a small amount of code. It could be a few lines of code. Then run the code. Use some print or console.log() statements to the show the results of your code execution to prove to yourself that your code does what you expect. If your code works then write a little more code and print out the results. If your code stops working correctly then you know where the problem is -- in the little amount of code you just wrote. Focus your attention on and increase your understand of that little bit of code you just wrote to find the source of the problem.

If you are handed code or script you are unfamiliar with and it doesn't work and you don't know why then you can use the "program a little at a time" technique to debug it. Create a brand new, blank script to start from. Copy a little amount of code from the start of the script you are debugging. Add some print/console.log statements and run the code. If it works then copy a little more code into your new script and run it. Keep doing this, little by little, until something breaks and then increasing your understanding of what breaks and fix it and continue programming little by little.
Also, make sure you understand any code you copy into your script. Perhaps the biggest sin in programming is writing or copying code you don't understand. Don't do that.

Follow the Logic

Sometimes code does not execute correctly because there is a logic error, meaning the code does something other than you think or expect. The way to solve this is to read the code of the script line by line starting at the beginning. Understand what each line does and see how each line logically follows the next line. This is executing a program in your mind.
Sometimes running the program in your mind as you read it is enough to notice that the code does something that does not make sense. Sometimes it is not enough because you don't understand the code well enough to do that. When it isn't enough to execute it in your mind then you can add print or console.log statements to the code to show the output and progress of the script. Check that the output corresponds to what you expect it to be. Prove to yourself that you understand what each line of code is doing by getting the expected output of print or console.log statements in your code.

Your script may call code or functions from another script or from a library or a third-party smart contract. Sometimes it is necessary when following the logic to jump to the source code of another script or library or smart contract to follow the logic there. This is necessary because the bug may be in the script or library that you are using or you need to follow the logic there so you understand what the script or library or smart contract is doing and how it affects the code you are writing.

Follow the logic can be used in combination with other debugging techniques like program a little at a time or divide or conquer.

A debugger can be used to follow the logic.

When things don't work follow the logic. The better you can follow the logic the more capable you are at finding out what is wrong.

Divide and Conquer

Sometimes you can have a bug or problem in your code but you don't know where it is. This can be solved by the divide and conquer approach. This approach is used to find where in code a problem is.

Comment out the last half of the script or code. Or add a statement in the middle of the code that kills the process. If the first half of the code runs correctly then you know that the bug/problem lies in the second half of the code. So if the first half of the code runs correctly then find the middle of the second half of the code and comment out the code from the middle to the end. Run the code. If the code runs correctly then you know the problem is in the commented out code. Or if running the code now shows up the bug then you know the section where the bug is and can further divide and conquer to further pin point it.

If the bug or problem shows up when running the first half of a script or code then you know the bug is in the first half of the code. Now you need to use the same approach on the first half of the code. Find the middle of the first half of the code and comment out from the middle to the end. See if the code works. If it does work then you know that the bug or issue is in the section that you just commented out and can further divide and conquer. If the code doesn't work then you know the problem or bug is in the first half of the code you just ran and you can further divide and conquer.

Once you have pin pointed much more closely where in the code the bug or problem is you can focus your attention there to find the bug or problem. You can write console.log statements to look at output to better understand what is happening. Look and increase your understanding of the section of the code where the bug is to find it and fix it.

There are other ways to debug by dividing and conquering. When you want to find out where a bug is look for ways to remove or divide code and run isolated code to see if the bug shows up.

Compare Code that Doesn't Work to Code That Works

If you have code that doesn't work but you don't know why and you have read the documentation about the code involved but it still doesn't work then one approach is to find code that does the same or similar thing that does work. Compare your code to code that works to find out what you are doing different.

There are many scripts and smart contracts in our repos that can be looked at. Google can be searched to find code that does something.

Change of Behavior

Sometimes code can be debugged by making changes to code and running it and observing changes or differences in results. This method of debugging gives you information about what the code is doing. It can give you clues to what is happening and the source of the problem. Remember that debugging is about increasing your understanding of the code and software so you can determine the cause of a problem.

Ethereum Diamonds Solve These Problems

Nick Mudge — Sat, 01 May 2021 22:38:43 +0000

Diamonds solve these problems:

The maximum size a smart contract can be on Ethereum is 24kb. But sometimes larger smart contracts are needed or desired. Diamonds solve that problem.
Provides a structure to systematically and logically organize and extend larger smart contract systems so they don't turn into a spaghetti code mess.
Provides fine-grained upgrades. Other upgrade approaches require replacing functionality in bulk. With a diamond you can add, replace, or remove just the functionality that needs to be added, replaced or removed without affecting or touching other smart contract functionality.
Provides a single address for a lot of smart contract functionality. This makes integration with smart contracts and user interfaces and other software easier.

If you plan to build a system that can grow or change over time then a diamond is a good choice because it enables a system to grow in a systematic, organized way.

The standard for diamonds is here: https://eips.ethereum.org/EIPS/eip-2535

Resources: https://eips.ethereum.org/EIPS/eip-2535#learning--references

How to Share Functions Between Facets of a Diamond

Nick Mudge — Wed, 31 Mar 2021 06:06:01 +0000

There are a number of ways to share or reuse functions between facets of an Ethereum diamond. The best way I have found is to write internal functions in Solidity libraries and call those functions in facets.

It is a common misunderstanding to think that all Solidity libraries are deployed. Only Solidity libraries with external functions must be deployed. Solidity libraries with only internal functions are not deployed -- they are added to the bytecode of contracts/facets that use them.

Also, only the bytecode from the internal functions that are actually used from a Solidity library are added to the bytecode of a facet. So it is possible to have large Solidity libraries that are utilized across facets without adding too much to the bytecode of facets that use them.

It is possible to use diamond storage and/or AppStorage in Solidity libraries which makes libraries even more useful and powerful.

Checkout Aavegotchi's libraries which are used across facets.

In addition to using internal functions in Solidity libraries EIP-2535 Diamonds describes a number of other ways functions can be shared or reused between facets:

Write internal functions in Solidity libraries and call those functions in facets. These functions can access diamond storage or AppStorage directly.
Copy internal function code in one facet to the other facet.
Put common internal functions in a contract that is inherited by multiple facets.
A type safe way to call an external function defined in another facet is to do this: MyOtherFacet(address(this)).myFunction(arg1, arg2)

A more gas-efficient way to call an external function defined in another facet is to use delegatecall. Here is an example of doing that:

DiamondStorage storage ds = diamondStorage();
bytes4 functionSelector = bytes4(keccak256("myFunction(uint256)"));
// get facet address of function
address facet = ds.selectorToFacet[functionSelector];
bytes memory myFunctionCall = abi.encodeWithSelector(functionSelector, 4);
(bool success, uint result) = address(facet).delegatecall(myFunctionCall);
require(success, "myFunction failed");

Instead of calling an external function defined in another facet you can instead create an internal function version of the external function. Add the internal version of the function to the facet that needs to use it.

AppStorage Pattern for State Variables in Solidity

Nick Mudge — Tue, 02 Feb 2021 05:34:00 +0000

I have written about the diamond storage pattern for organizing contract state variables in proxy contracts and diamonds.

As a quick refresher, a state variable or storage layout organizational pattern is needed when writing proxy contracts or diamonds in Solidity because Solidity's builtin storage layout system doesn't work for them.

Diamond storage is particularly good for isolating or compartmenting state variables to specific facets or functionality. This is great for creating modular facets that can be understood as their own units and be added to diamonds. A diamond with a lot of functionality is well organized and understandable if each of its facets can be understood in isolation. Diamond storage helps make that possible.

However, I have found that when building applications with diamonds I often want to share state variables specific to my application with facets that are specific to my application. It gets tedious to call a diamondStorage() function in every function that I want to access state variables.

I have successfully utilized a new pattern in contracts called AppStorage. It is a nicer and more convenient way to access application specific state variables that are shared among facets.

The pattern works in the following way:

1) Define a struct called AppStorage that contains all the state variables specific to your application and that you plan to share with different facets. Store AppStorage in a file. Any of your facets can now import this file to access the state variables.

// AppStorage.sol
struct AppStorage {
  uint256 secondVar;
  uint256 firstVar;
  uint256 lastVar;
  ...
}

2) In a facet that imports the AppStorage struct declare an AppStorage state variable called s. This should be the only state variable declared in the facet. Here's an example:

import "./AppStorage.sol"

contract StakingFacet {
  AppStorage internal s;
  ...

Now in your facet you can access all the state variables in AppStorage by prepending state variables with s.. Here's a simple example:

import "./AppStorage.sol"

contract StakingFacet {
  AppStorage internal s;

  function myFacetFunction() external {
    s.lastVar = s.firstVar + s.secondVar;
  }

This is a very nice convention because it makes state variables concise, easy to access, and it distinguishes state variables from local variables and prevents name clashes/shadowing with local variables and function names. It helps identify and make explicit state variables in a convenient and concise way. It is also a little more gas efficient than diamond storage access.

Since AppStorage s is the first and only state variable declared in facets its position in contract storage is 0. This fact can be used to access AppStorage in Solidity libraries using diamond storage access. Here's an example of that:


library MyLib {
  function appStorage() 
    internal 
    pure 
    returns (AppStorage storage ds) 
  {    
    assembly {
      ds.slot := 0
    }
  }

  function someFunction() internal {
    AppStorage storage s = appStorage();
    ... do stuff
  }
}

AppStorage s can be declared as the one and only state variable in facets or it can be declared in a contract that facets inherit.

AppStorage won't work if state variables are declared outside of AppStorage and outside of Diamond Storage. It is a common error for a facet to inherit a contract that declares state variables outside AppStorage and Diamond Storage. This causes a misalignment of state variables.

One downside is that state variables can't be declared public in structs so getter functions can't automatically be created this way. But I think it is better anyway to make your own getter functions for state variables because its more explicit.

The GHSTStakingDiamond is a good example of a diamond that uses the AppStorage pattern.

For a large, in production, successful example of using the AppStorage pattern, checkout the Aavegotchi diamond. The AppStorage struct is defined in this file: LibAppStorage.

Notice that the AppStorage struct utilizes other structs in arrays and mappings.

Addressing Josselin Feist's Concern's of EIP-2535 Diamonds

Nick Mudge — Sun, 01 Nov 2020 02:14:00 +0000

Josselin Feist, from Trail of Bits, wrote an article with a number of concerns about EIP-2535 Diamonds and an implementation of it. I'll address concerns he brought up and correct some things along the way.

One thing to mention is that EIP-2535 Diamond and its implementations and tools and information is a community effort. There is a growing group of people making the standard and its growing ecosystem what it is.

I invite Trail of Bits and other people to help fill in the gaps of information, tests, procedures, ideas and tools to help make diamonds and upgrades better and easier to use for everyone.

The Diamond Discord is here: https://discord.gg/kQewPw2

EIP-2535 Diamonds Is Not Just For Upgrades

Josselin's article describes EIP-2535 Diamonds as an "upgradeable standard" only. This is a fundamental misunderstanding of the standard. EIP-2535 Diamonds is more than just a standard for upgrading smart contracts. In addition “upgradeability” is an optional feature of the standard.

EIP2535 Diamonds is a general purpose smart contract framework for building multi-contract systems. Diamonds can be upgradeable or not. Diamonds provide a way to have smart contract functionality that exceeds the 24kb smart contract size limit at a single Ethereum address. It provides a standard way to organize the source code of a large smart contract system. It provides a mechanism for on-chain code reuse and modularity. It provides mechanisms for transparency and tooling.

Yes, it also enables upgrades (if desired), in such a way that a project can be deployed and then be extended as needed in a systematic and organized way, without limit. And diamonds provide fine-grained upgrades so code that doesn't need to be changed doesn't have to be redeployed.

Josselin's Article Is Biased Against EIP2535 Diamonds With False Data

Josselin's article is not an objective, honest review of EIP2535 Diamonds. It contains outdated code and false data. His article brings up security concerns that are addressed and handled by the standard and related diamond documentation.

A prominent and obvious technical advantage of EIP2535 Diamonds is that it provides a way for a smart contract to exceed the 24kb smart contract size limit. Josselin's article fails to mention that and other benefits of the standard.

About Upgrades

Josselin says an upgradability standard should have the following things:

A clear, simple implementation. Standards should be easy to read to simplify integration with third-party applications.

Understood, this has been done with the diamond-1-hardhat implementation.

A thorough checklist of upgrade procedures. Upgrading is a risky process that must be thoroughly explained.

A list of things to watch out for and rules to follow to ensure safe upgrades has been written and exists here: https://eip2535diamonds.substack.com/p/diamond-upgrades

On-chain mitigations against the most common upgradeability mistakes, including function shadowing and collisions. Several mistakes, though easy to detect, can lead to severe issues.

The diamond reference implementations contain no function shadowing and provides a solution for handling that. The standard prevents function selector clashes. Diamond storage helps prevent contract storage corruption or collisions. Additional tools and ways to do this are welcome. There is also information about upgradeable contracts and their pitfalls that can be found online.

I have written many articles (and there are others online) about how to correctly and safely write smart contracts and diamonds. These can be found here: https://eip2535diamonds.substack.com/ and here: https://dev.to/mudgen

A list of associated risks. Upgradeability is difficult; it can conceal security considerations or imply that risks are trivial.

A list of things to watch out for and rules to follow to ensure safe upgrades has been written and exists here: https://eip2535diamonds.substack.com/p/diamond-upgrades

Also this blog post about Pros and Cons of diamonds is recommended reading: https://eip2535diamonds.substack.com/p/answering-some-diamond-questions

Tests integrated with the most common testing platforms. The tests should highlight how to deploy the system, how to upgrade a new implementation, and how an upgrade can fail.

The diamond reference implementations have a suite of tests to test them. The reference diamond implementations and other implementations show ways to deploy and upgrade diamonds.

hardhat-deploy provides builtin support for deploying and upgrading diamonds and includes documentation about it.

Contract Standardization

One of the main purposes of a contract standard is to provide standard interfaces with implementation details so that implementations can be made that interoperate with other contracts and tools and software.

EIP-2535 Diamonds does this by providing a standard function diamondCut for upgrades, a standard event DiamondCut for recording what functions are added/replaced/removed and 4 standard functions that are used to show what functions a diamond has.

The standard provides information about how to implement these functions and other useful information. All of this is useful and being used in development and production successfully.

Diamond Implementation

Josselin shows part of an implementation of the diamondCut function that is used for upgrades. In several places he mentions that gas-optimizations make the code more complicated:

Additionally, much of the code complexity is due to optimization in locations that don’t need it.

The gas-optimizations that were made in that implementation save gas and it is really up to the people building an application whether or not the gas optimizations are needed or wanted or not.

People who aren't interested in gas-optimizations can use the much simpler diamond-1-hardhat implementation.

Josselin's article is badly out of date. The code shown for the upgrade function was removed and replaced by simplified code 7 weeks before the date Josselin's article was published. The code he shows does not exist in any of the current diamond reference implementations. I have asked Josselin to update his article with code from a current implementation, but he has not done that.

Josselin was hired from Trail of Bits back in 2020 to help us find bugs and improve our code. And he did and we fixed and improved things. Then he published an article featuring our old, unfixed code to represent EIP-2535 Diamonds. To be accurate he should provide information and code for current implementations.

Also note that the EIP-2535 Diamond is a smart contract standard. It is not an implementation. There can be good or bad implementations of it. Josselin's article conflates an outdated implementation with the standard.

Diamond Terminology

Diamond A contract that gets its external functions from other contracts called facets. A diamond has a fallback function that delegates function calls to facets. A diamond implements the Specification section of EIP-2535 Diamonds.
Facet A contract that supplies external functions to a diamond. In the diamond industry a facet is a side of a diamond.
Loupe Four standard functions that show what functions and facets a diamond has. In the diamond industry a loupe is a magnifying glass that is used to look at diamonds.
Upgradeable Diamond A diamond that has the diamondCut external function so functions can be added/replaced/removed.
Finished Diamond A diamond that was an upgradeable diamond but then its diamondCut function was removed so functions can no longer be add/replaced/removed. It can't be upgraded anymore.
Single Cut Diamond A diamond that adds all of its functions and facets in its constructor function, and does not add the diamondCut or other upgrade function. A single cut diamond is fully formed upon deployment, is immutable and can't be upgraded.

Josselin says:

EIP 2535 introduces “diamond terminology,” wherein the word “diamond” means a proxy contract, “facet” means an implementation, and so on. It’s unclear why this terminology was introduced.

Let me clear this up.

Originally EIP-2535 Diamonds was called the Transparent Contract Standard, but that name clashed with something from OpenZeppelin and they asked me to change the name.

When making transparent contracts it was noticed that each of its facets/implementations could handle a different aspect or set of functionality and it was useful to organize code this way. It was like each facet/implementation was a different side or facet of a transparent contract.

Physical diamonds have facets that share a common center. EIP-2535 diamonds have facets that share the same contract storage space and a common Ethereum address.

The terminology helps conceptualize how diamonds work and how to build applications with them.

The Loupe Functions

The loupe functions are four standard functions defined by EIP-2535 Diamonds that return what functions a diamond has and the facet addresses they come from.

The verified source code of a diamond does not show what functions it has or where it gets them. And a diamond's ABI doesn't include what functions it has. To help get this information the DiamondCut event or the loupe functions are used.

The loupe functions are being used successfully by the application https://louper.dev/ to display and make diamonds transparent. It is like etherscan but for diamonds.

One of the loupe functions is facetFunctionSelectors. Josselin says this about it:

facetFunctionSelectors returns all the function selectors of an implementation. This information will only be useful for off-chain components, which can already extract the information from the contract’s events. There’s no need for such a feature on-chain

An important reason the loupe functions (including facetFunctionSelectors) exist is because functions on smart contracts are more reliable than events. These functions make diamonds robust. This ensures their transparency and interoperability with software.

Some event systems fail, some are inconsistent and fragile. Events work better on some blockchains than others. For example on Polygon where block times are fast querying events on nodes will fail if trying to query too far back in time. Basing a system on an unreliable event system makes a system also unreliable or harder to deal with or program to make reliable. This is why diamonds use on-chain loupe functions for transparency. It is robust.

Currently the loupe functions are being used in unit tests to test diamonds. They are being used in code that deploys and upgrades diamonds. They are used by hardhat-deploy to deploy and upgrade diamonds.

It is true that DiamondCut events can be queried and processed to get the same data that the loupe functions return. This means there are two ways to get data about diamonds. Both ways can get data about diamonds and either way can be used.

So why have two ways? Because in some cases it will be easier and better to use function calls, and in other cases it will be easier and better to use events.

For example a blockchain explorer website may want to use events to show facet and function information about diamonds because it is already using event tooling and infrastructure to capture and process events for contracts.

An upgrade script may want to check that functions don't already exist in a diamond before trying to add them. Using the facets loupe function for this is simple and easy and reliable.

A simple static website can show information about any diamond by calling the loupe functions and then using the returned data to query services like Etherscan to get and show verified source code, ABI information and other information. This could be done with events, but it would be easier, simpler and probably run faster if done with loupe functions.

Having loupe functions and events give people and software choice to use what is best for their use.

There is also a question about reliability. Querying and processing events is not as reliable, available and performant as contract function calls. Likewise, a connection to an Ethereum node may not always be available in every system but there may be a connection to an events database that can be used to retrieve diamond information. Having both events and loupe functions makes diamonds robust.

The loupe functions add some complexity to diamonds. Generally it is a good idea to push complexity out of contracts into off-chain software. But every case is different and requires evaluation. In the case of diamonds there is another principle which is optimize for simplicity for common use cases. A diamond is implemented and deployed once but will be queried many times and may be used by or integrated with a lot of software. The loupe functions are a simple, reliable and direct way to query diamonds and integrate with software. The loupe functions are read-only, tested and security audited. The loupe functions can simply be added and used in every diamond without touching application specific smart contract code.

In addition the loupe functions can be deployed once and be reused on-chain by many diamonds.

Someone suggested the idea of making the loupe functions optional in the standard. The problem with that is it would break interoperability. Some diamonds would work with some software and not with other software.

I wrote another article about loupe functions that provides more information: Diamonds Loupe Functions

Addressing False Data

Many Interfaces

While the proposal only needs a lookup table, from the function signature to the implementation’s address, the EIP defines many interfaces that require storage of additional data:

This statement is a lie. It is easily verifiably false. You can easily fact check it yourself.

Josselin says the EIP "defines many interfaces that require storage of additional data". If you look at EIP-2535 you will see that the EIP only defines two interfaces in the entire standard. So unless you think "many" means two the statement is false. But still, only one of the interfaces requires storing additional data besides a lookup table. One definitely does not mean "many".

The IDiamondLoupe interface requires storing function selectors in an array and their positions in the array. That is all the additional data that needs to be stored.

Why does Josselin lie about this? If it is just a mistake then why doesn't he fix it?

In general I suggest that you verify the technical details for yourself with regards to Josselin's article. The diamond reference implementations can be found from here: https://github.com/mudgen/diamond

Safe Struct Storage Pointers with Diamond Storage

EIP-2535 Diamonds shows and encourages a contract storage technique called Diamond Storage.

Diamond Storage enables different facets to share or use their own contract storage space within a diamond and access it with a namespace. The namespace is a unique, human-readable and meaningful string that represents the area of data. For example "com.mywebsite.myproject.mytoken". The string is hashed and used as a pointer to a struct in contract storage. More information about this and examples are in this article: How Diamond Storage Works.

There is no risk of accidentally or maliciously colliding diamond storage with other contract storage because of using hashes of unique and meaningful strings for struct storage pointers.

OpenZeppelin plans to use Diamond Storage in their next version of upgradeable contracts.

Josselin's article shows an example of colliding contract storage data. However his storage pointer is not a hash of a unique and meaningful string. His storage pointer is not even a hash of a string but is a hash of a hex literal of a specific but random looking value. It is easy to see that it is not a namespace to an area of contract storage.

Malicious storage pointers are obvious and easy to spot. Here is a comparison:

Namespace to an area of contract storage:

bytes32 storagePosition = keccak256("myproject.mytoken")

Malicious pointer that collides with existing contract storage data:

bytes32 storagePosition = keccak256(
  hex"78c8663007d5434a0acd246a3c741b54aecf2fefff4284f2d3604b72f26"
);

Someone adding code to their contract or diamond with a malicious pointer means that the person is adding code that they didn't look at or understand. If that is the case then any kind of malicious code could be added.

Diamond Storage Not Required

The EIP proposes that every implementation have an associated structure to hold the implementation variables, and a pointer to an arbitrary storage location where the structure will be stored.

EIP-2535 Diamonds does show, explain and encourage using diamond storage. But it does not require it.

Diamonds and facets are free to use or mix any contract storage strategy such as inherited storage etc.

To comply with the standard and be a diamond an implementation must implement the Specification section of the standard, which does not mandate any contract storage strategy or mechanism.

Function Shadowing

Upgradeable contracts often have functions in the proxy that shadow the functions that should be delegated. Calls to these functions will never be delegated, as they will be executed in the proxy. Additionally, the associated code will not be upgradeable.

None of the diamond reference implementations have any functions shadowing any other functions. Function shadowing is not possible in diamonds that follow the standard.

It is possible to have external functions directly in a diamond in a safe way. They are called immutable functions in EIP-2535 Diamonds. They are functions that can't be replaced or removed. The diamond loupe functions provide information about them and they are emitted in the DiamondCut event. In the reference implementations calling diamondCut to replace or remove immutable functions will revert. See the standard for more info.

Contract Existence Check

Another common mistake is the absence of an existence check for the contract’s code. If the proxy delegates to an incorrect address, or implementation that has been destructed, the call to the implementation will return success even though no code was executed (see the Solidity documentation). As a result, the caller will not notice the issue, and such behavior is likely to break third-party contract integration.

Note: Josselin's article shows a fallback function without the contract code check he talks about.

The fallback function in the diamond reference implementations do not check for contract code. It does not do this because of the gas cost and because the diamondCut function prevents any functions from being added from facets that do not have code.

The rule which is documented is don't write selfdestruct in facet source code and don't use facets with selfdestruct in them unless it specifically makes sense for some use case. It is unnecessary and useless (and costs gas) to check for contract existence on contracts that can never be deleted.

OpenZeppelin's proxy contracts also do not perform the check in the fallback function. Like the diamond reference implementations they prevent implementation contracts from being added that don't have code.

EIP-2535 Diamonds does not prevent people from adding this check in the diamond fallback function so it can be added by people who want it. Contract existence check in the diamond fallback function is not part of the EIP2535 Diamonds standard so it is up to the implementer to implement it or not.

Diamond Loupe Functions

Nick Mudge — Thu, 08 Oct 2020 20:34:36 +0000

A loupe is a magnifying glass that is used to look at diamonds.

In EIP-2535 Diamond Standard a loupe is four standard read-only functions that tell you what functions and facets are provided by a diamond.

A facet is a smart contract that supplies diamonds with external functions.

If you look at the source code of a diamond it will show a fallback function. It doesn't show what external functions it has. To get that information you need to call one or more of the four standard, read-only loupe functions.

Here are the four standard read-only loupe functions:

facetAddresses() Returns all the facet addresses used by a diamond.
facetAddress(bytes4 _functionSelector) Returns the facet address that implements a functions.
facetFunctionSelectors(address _facet) Returns all the functions used by a diamond that come from a particular facet.
facets() Returns an array consisting of all the facet addresses and functions used by a diamond.

The loupe functions can be used for many useful things including:

Use the facet addresses to query services like Etherscan to retrieve and show all the source code used by a diamond.
Retrieve ABI info for a diamond.
Tools and programming libraries use them to deploy diamonds.
Tools and programming libraries use them to upgrade diamonds.
User interfaces use them to show information about diamonds.
User interfaces use them to enable users to show and call functions on diamonds.
Tests use them to verify that correct functions and facets are being used correctly.

The loupe functions make diamonds transparent. This is important for understanding, showing and using exactly the functionality provided by diamonds.

They are important for security to verify that diamonds are used correctly and not mistakenly or maliciously.

They are important for testing to ensure that diamonds contain the correct functions and facets and to test code that uses diamonds.

Another Way. Transparency With Events

The loupe functions are not the only way to find out what functions and facets a diamond has.

Anytime a function/facet is added/replaced/removed on a diamond the DiamondCut event is emitted that records exactly what changed in a diamond.

All the DiamondCut events for a diamond can be queried and an algorithm executed to determine what facets and functions currently exists for a diamond.

You might wonder why we have two ways to do the same thing.

First, the DiamondCut event provides something that the loupe functions don't: A historical record of all upgrades done on a diamond. The historical record can be used to verify the state, correctness and security of a diamond over time.

So you might wonder why have loupe functions when events can be used to get the current functions and facets used by a diamond.

Why Have Loupe Functions When You can Use Events?

To be clear, this isn't a debate of events vs external functions. The DiamondCut events are a given and there is no restriction on their use. The debate is this: Why have events and loupe functions?

Let's look at the tradeoffs of having/not having loupe functions.

Pros

More choice People that want to use event systems, such as the The Graph, or event database systems, to determine the contents of diamonds can do so. People who want to call the loupe external functions on diamonds to get the same data, can do so. Having events and the loupe functions make diamonds easier to use for more people because they can choose how to get the data.
Easier adoption by services and tools Services such as Etherscan, blockchain explorers, web applications, programming libraries and other software may prefer or require the use of events or external functions. Having both options increases the capability to integrate with tools and services.
More transparent, reliable and available If an event system is not available or up-to-date or is slow or has any other problem, then the loupe functions can be used. If external functions can't be called in a particular context then an event system or database can be used. Having both methods available increases the reliability, availability and transparency of diamonds.
Simple and easy The loupe functions are simple and easy to use. They are regular external contract functions that are called with regular client contract libraries like web3.js or ethers.js. No special custom software or third party software or library or dependency is needed to call the loupe functions, since they are regular contract functions.

Keeping data, code and logic within the diamond loupe functions simplifies client software that needs the data.

Software that uses events has to rely on custom software or a third-party library to query all the DiamondCut events and execute logic to determine the current functions and facets.
Secure The code for the diamondCut function and the diamond loupe functions is carefully looked at by Solidity developers and security experts. These functions are simple enough for a competent Solidity developer to read and understand.
In theory the loupe functions can be executed on-chain. In theory the loup functions in the diamond-3 implementation are gas-efficient enough to be called and used in transactions.

Cons

Loupe functions add code to a diamond.

However a DiamondLoupeFacet only needs to be deployed once and can be reused by many diamonds.

The diamond-1 implementation does the following to implement loupe functions:

Stores an array of function selectors. When a new function is added its selector is added to the function selectors array. When a function is removed its selector is removed from the function selectors array.
The loupe functions are implemented like this:
1. facetAddress(bytes4 _functionSelector) The facet address that is associated with the _functionSelector is found and returned.
2. facetAddresses The function selectors are looped through to find all the facet addresses used by the diamond. An array of facet addresses is returned.
3. facetFunctionSelectors(address _facet) The function selectors are looped through to find all the selectors that belong to the facet specified by _facet. An array of selectors is returned.
4. facets() The function selectors are looped through to find all the facet addresses and function selectors used by the diamond. These are returned.

The diamond-2 implementation is implemented the same way as diamond-1 except that its code is more complicated because it is gas-optimized.

The diamond-3 implementation does the following to implement loupe functions:

Stores an array of facet addresses. For each facet address an array of function selectors is stored. When a new facet is added it is added to the array of facet addresses. When a facet is removed it is removed from the array of facet addresses. When a function is added its selector is added to an array of function selectors. When a function is removed its selector is removed from an array of function selectors.
1. The loupe functions are very simple and implemented like this:
  1. facetAddress(bytes4 _functionSelector) The facet address that is associated with the _functionSelector is found and returned.
  2. facetAddresses The array of facet addresses is returned.
  3. facetFunctionSelectors(address _facet) The array of function selectors for the facet is returned.
  4. facets() The array of facet addresses is looped through to get each facet address and its function selectors. These are returned.

Note that the loupe functions are read-only and are isolated in their own facet. They do not affect other facets of a diamond. Because of this the loupe functions do not increase complexity of the main application code or other facets.

Loupe functions require more gas for adding/replacing/removing functions Loupe functions require that some data be stored in the diamond so that function selectors and facet addresses can be retrieved. The diamond-2 implementation is the most efficient for adding/replacing/removing functions. It stores 8 function selectors in a single 32-byte contract storage slot and avoids storage reads and writes.

I don't have gas metrics for diamond implementations that don't have loupe functions. But I do have some gas metrics for diamond implementations that do have loupe functions.

diamond-1 implementation To add 20 functions from a single facet costs 617,348 gas.
diamond-2 implementation To add 20 functions from a single facet costs 536,246 gas.
diamond-3 implementation To add 20 functions from a single facet costs 683,000 gas.

Overall

I think having events and loupe functions is worth it. I've been working on EIP-2535 Diamonds and diamond implementations to be useful for myself and you and others. So I am interested in your input about this.

How Diamond Storage Works

Nick Mudge — Fri, 18 Sep 2020 12:57:03 +0000

Diamond storage is a contract storage strategy that is used in proxy contract patterns and diamonds.

It greatly simplifies organizing and using state variables in proxy contracts and diamonds.

Diamond storage relies on Solidity structs that contain sets of state variables that are easy to read and write.

A struct can be defined with state variables and then used in a particular position in contract storage. The position can be determined by a hash of a unique string. The string acts like a namespace for the struct. For example a diamond storage string for a struct could be com.mycompany.projectx.mystruct. That will look familiar to you if you have used programming languages that use namespaces.

Namespaces are used in some programming languages to package data and code together as separate reusable units. Diamond storage packages sets of state variables as separate, reusable data units in contract storage.

Let's look at a simple example of diamond storage:

library MyStructStorage {
  bytes32 constant MYSTRUCT_POSITION = 
    keccak256("com.mycompany.projectx.mystruct");

  struct MyStruct {
    uint var1;
    bytes var2;
    mapping (address => uint) var3;
  }

  function myStructStorage()
    internal 
    pure 
    returns (MyStruct storage mystruct) 
  {
    bytes32 position = MYSTRUCT_POSITION;
    assembly {
      mystruct.slot := position
    }
  }
}

The diamond storage defined above can be used like this:

function myFunction()
  external
{
  MyStructStorage.MyStruct storage mystruct =
    MyStructStorage.myStructStorage();

  mystruct.var1 = 10;
  uint var3 = mystruct.var3[address(this)];

  // etc
}

Instead of using a hash of a string other schemes can be used to create random positions in contract storage. Here is a scheme that could be used:

bytes32 constant MYSTRUCT_POSITION = 
    keccak256(abi.encodePacked(
      ERC1155.interfaceId, 
      ERC1155.name, 
      address(this)
    ));

Diamond storage is easy to use correctly, but like any tool it needs to be used correctly.

Don't make these mistakes:

Do not use the same namespace string for different structs. Because the two structs will overwrite each other in storage.
Don't do the following: Deploy a contract that uses a particular struct. Then upgrade your contract with a modified version of the struct that is stored at the same location and with new state variables added or state variables removed in the beginning or middle. This is similar to storing two different structs at the same location in contract storage. However it is safe to add new state variables to the end of structs that are already being used.
Don't put structs directly in structs unless you are sure you won't want to add new state variables to the end of the inner structs. You won't be able to add new state variables to inner structs in upgrades. But you can put structs in mappings, and still extend the structs in the future. To understand this more read about how state variables are laid out in contract storage here: https://docs.soliditylang.org/en/latest/internals/layout_in_storage.html

These mistakes are easy to avoid by being aware of them and keeping track of what namespaced strings are used. Also a naming convention can be used to give structs unique strings.

Here are some additional references that cover diamond storage:

Diamond storage is also called common storage. Smart contract developer Jules Goddard writes about it:

Should Diamond Loupe Functions Return Tightly Packed Data?

Nick Mudge — Mon, 31 Aug 2020 22:03:55 +0000

Quick Background Material

Diamond: A diamond is a contract with functionality from multiple contracts that can share internal functions, libraries and state variables. Diamonds implement EIP-2535 Diamond Standard.

Facet: The word facet comes from the diamond industry. It is a side, or flat surface of a diamond. A diamond can have many facets. In EIP-2535 Diamond Standard a facet is a contract that supplies a diamond with external functions. A diamond gets its external functions from facets.

Loupe: A loupe is a magnifying glass that is used to look at diamonds. In EIP-2535 Diamond Standard the loupe is four standard functions that return what functions and facets a diamond currently has.

Tightly Packed Encoding vs ABI Encoding

Currently, three of the diamond loupe functions return a tightly packed bytes array.

The great thing about a tightly packed bytes array is that it doesn't waste bytes. A tightly packed bytes array doesn't contain any useless bytes containing the value 0, unlike ABI encoded arrays.

The alternative to a tightly packed bytes array is an ABI encoded array.

ABI encoded arrays are the default data encoding used by Solidity contracts. They are more standard than tightly packed bytes arrays and are easy to use. You or something has to encode/decode tightly packed bytes arrays. Ethereum contracts and tools handle all the encoding/decoding of ABI encoded arrays for us.

But ABI encoded arrays are very inefficient, wasting bytes. ABI encoding uses 32 bytes for many values when that many bytes are not needed.

Let's look at some loupe functions:

facetAddresses loupe function

The facetAddresses loupe function returns a bytes array of all facet addresses currently used by a diamond. The bytes array is tightly packed so every byte in the array is used.

Instead of returning a bytes array of addresses the facetAddresses function could return an ABI encoded array of addresses.

Let's compare the two approaches: Let's say that a diamond has 10 facet addresses. An address is 20 bytes in size.

Using a tightly packed bytes array the size is 200 bytes. Because 10 * 20 = 200.
Using an ABI encoded array of addresses the size is 320 bytes. Because 32 * 10 = 320. Each value in the array has 32 bytes and only 20 bytes of each value is used. For many values ABI encoding uses 32 bytes whether they need them or not.

facetFunctionSelectors loupe function

The loupe function facetFunctionSelectors returns a tightly packed bytes array of all function selectors currently used by a diamond.

Instead of using a tightly packed bytes array it could return an ABI encoded array of bytes4 values.

Let's compare the two approaches: Let's say that a diamond has 100 function selectors. Each function selector is 4 bytes in size.

Using a tightly packed bytes array the size is 400 bytes. Because 100 * 4 = 400.
Using an ABI encoded array of bytes4 the size is 3200 bytes. Because 100 * 32 = 3200. Each value in the array has 32 bytes even though only 4 of the bytes are used in each value.

facets() loupe function

This function returns an array of tightly packed bytes arrays. Each bytes array holds a facet address and the function selectors associated with the facet address. I'll let you or someone else figure out or make a comparison of tightly packed encoding and ABI encoding for this one.

So What Should Diamonds Use?

Tightly packed encoding is efficient.

ABI encoding is inefficient.

But ABI encoding is very standard and used by all Ethereum tools and software. ABI encoding is easier to use and write software with. It is nice. Maybe that matters more than the efficiency of data encoding.

I originally chose tightly packed bytes array return values for EIP-2535 Diamond Standard because I envisioned that there would be thousands of diamonds in the world and off-chain client software continuously calling the loupe functions to provide information about diamonds. Tightly packed bytes arrays would reduce bandwidth costs and perhaps offer other scaling advantages. But in a world where computing resources are already cheap and getting cheaper, maybe it doesn't matter. Maybe easier software now matters more.

Easier to write software can have less bugs.

So what do you know and think? Should EIP-2535 Diamond Standard be changed to use ABI encoded array return values for the loupe functions? Or should it continue to use tightly packed bytes arrays?

UPDATE 6 September 2020

Thanks to all the developers who provided their feedback and input about whether the loupe function return values should be ABI encoded or tightly packed.

It was interesting to discover that developers were as mixed about it as myself.

I removed the tightly packed encoding of loupe function return values from EIP-2535 Diamond Standard. Loupe function return values now use ABI encoding.

I made this change because at this time the data efficiency of tightly packed encoding isn't needed. In the future if it is needed or more useful then a new standard for efficient loupe functions can be written and used. In addition, at any time developers can write their own diamond related information retrieval functions with whatever data encoding efficiency they desire and use them while at the same time providing the four standard loupe functions.

ABI encoded return values are a bit easier to work with, a bit easier to learn and understand. In addition, something unexpected was discovered.

Recent research has shown that it is possible to make all the diamond loupe functions gas-efficient enough to be executed in on-chain transactions. ABI encoding simplifies the code to do this and may save gas.

Currently the diamond reference implementation is gas-optimized to use the least gas for adding/replacing/removing any number of functions. The loupe functions in the diamond reference implementation are great for off-chain software to use to show what functions and facets a diamonds use. But it costs too much gas to call the loupe functions in on-chain transactions. But it doesn't have to be this way.

It is possible to instead make the loupe functions gas-optimized for execution in on-chain transactions.

It is also possible to make the loupe functions gas-optimized for adding/replacing/removing functions and for execution in on-chain transactions. It would require a lot of code and be complicated but I think it is worth doing.

I think it would be interesting and awesome to have gas-efficient diamonds where you can query and process its functions in on-chain transactions.

DEV Community: Nick Mudge

Understanding EIP-2535 Diamonds: The Future of Smart Contract Design

What is EIP-2535?

Key Components of EIP-2535

Benefits of EIP-2535

How to Use EIP-2535 Diamonds

1. Setting Up Your Environment

2. Create Your Diamond Contract

3. Define Your Facets

4. Deploy Your Diamond and Facets

5. Interact with Your Diamond

Conclusion

How to solve OpenSea's "We couldn’t find this contract." error for ERC721 Validation

How OpenSea ERC721 Validation Works

When OpenSea ERC721 Validation Fails

Solidity: AppStorage for distinguishing state variables and preventing name clashes

How Diamond Upgrades Work

How Function Calls Work

How Diamond Upgrades Work

How to Debug Code & Debugging Techniques

Why is being good at debugging important?

How do you actually debug?

Lacking Knowledge

Oversight

Debugging Techniques

Look at the Error Message

Program a Little at a Time

Follow the Logic

Divide and Conquer

Compare Code that Doesn't Work to Code That Works

Change of Behavior

Ethereum Diamonds Solve These Problems

How to Share Functions Between Facets of a Diamond

AppStorage Pattern for State Variables in Solidity

Addressing Josselin Feist's Concern's of EIP-2535 Diamonds

EIP-2535 Diamonds Is Not Just For Upgrades

Josselin's Article Is Biased Against EIP2535 Diamonds With False Data

About Upgrades

Contract Standardization

Diamond Implementation

Diamond Terminology

The Loupe Functions

Addressing False Data

Many Interfaces

Safe Struct Storage Pointers with Diamond Storage

Diamond Storage Not Required

Function Shadowing

Contract Existence Check

Diamond Loupe Functions

Another Way. Transparency With Events

Why Have Loupe Functions When You can Use Events?

Overall

How Diamond Storage Works

Should Diamond Loupe Functions Return Tightly Packed Data?

Quick Background Material

Tightly Packed Encoding vs ABI Encoding

facetAddresses loupe function

facetFunctionSelectors loupe function

facets() loupe function

So What Should Diamonds Use?

UPDATE 6 September 2020

Find a job developing smart contracts using Solidity at delegatecall.careers