DEV Community: flex

SUBFROST: Utilizing BTC in Metaprotocol Space

flex — Thu, 12 Dec 2024 00:18:09 +0000

tldr: native Bitcoin restaking protocols are coming

Over the past several years, a great deal of my research and development effort has been in synthetics, specifically ones backed by Bitcoin. Such assets allow us to utilize native Bitcoin currency within the greater ecosystem of blockchain finance, and are of paramount importance. Generally, projects which aspire to connect Bitcoin to external consensus engines accomplish this using a multisignature custody mechanism, with supplementary incentive mechanisms which, theoretically, create decentralized governance and oversight.

Developing a system like this can be burdensome, but it can be accomplished. More often it is not without some tradeoffs or limited trust assumptions. The final form of such a product should be thought of as an L0, since in spirit it should belong to a consensus engine that allows us to transfer Bitcoin into another representation on a separate L1 or L2.

Projects in the Bitcoin L2 space often feature an L0 subsystem of some kind to facilitate bridging of value on and off an off-chain consensus program.

BTC on ALKANES

Unlike a scenario where we bridge to other blockchain systems or EVM, if we want to bridge to/from a metaprotocol on the Bitcoin L1 layer, we get the benefit of a shared space for asset representations, which coexist on the same L1 data layer.

On ALKANES, where we have WASM smart contract assets, we can have something like this on Bitcoin L1:

impl Token for SyntheticBitcoin {
    fn name(&self) -> String {
        String::from("SUBFROST BTC")
    }
    fn symbol(&self) -> String {
        String::from("frBTC")
    }
}

Source: https://github.com/subfrost/frBTC/tree/main/alkanes/fr-btc/src/lib.rs

This could get good.

In fact, if we are bridging Bitcoin to the ALKANES metaprotocol to a wrapped representation, we can actually design an alkane smart contract for the synthetic, where Bitcoin can be wrapped atomically and without trust assumptions. Since the action of wrapping Bitcoin to an alkane happens as part of a normal protorunes protocol message, it can be composed into other intents which actually use the Bitcoin synthetic and do something with it, all in the same Bitcoin transaction.

Additionally, the alkane for our synthetic can perform all the accounting necessary for a federated multisignature custodian to handle unwrapping back to Bitcoin. Even if we cannot actually unlock Bitcoin spends from the metaprotocol layer, we can use the metaprotocol to signal the signing group to perform this function.

That's a little abstract, but the easy way to put it is that wrapping Bitcoin to an alkane is instant, and unwrapping requires the decentralized multisig. The decentralized multisig reads directly from the alkane to know who burns synthetics and should receive Bitcoin.

SUBFROST is what we call this decentralized custodian and can exist as a very lightweight layer on top of an ALKANES indexer, which simply listens for new blocks and coordinates a small ring of signers which custody and redeem synthetic metaprotocol BTC back to native BTC currency. The ALKANES metaprotocol does all the work for us, so we do not need a heavy-duty consensus engine to coordinate the active group of signers to agree on what messages should be signed over the p2p channel.

Signers just run the ALKANES indexer and SUBFROST software, and thus the consensus model for our L0 exists purely as an extension of the Bitcoin protocol itself.

This is how we can activate Bitcoin in decentralized programs of any kind, to the limit of what we can develop on the WASM execution layer.

frBTC Usage

The repository for frBTC (SUBFROST BTC) is available at:

https://github.com/subfrost/frBTC

Examples of minting and burning frBTC are available in the test suite, but a web application will be available for ease of access to synthetics.

A frBTC asset challenges the problem surface of building on Bitcoin L1 as we have traditionally seen it:

In a single transaction, it becomes possible to:

Swap BTC to a Bitcoin native stablecoin asset directly via AMM, at the desired slippage tolerance.
Deposit BTC into a privacy pool, similar in function to what we see possible on Zcash
Use BTC as collateral to mint synthetic stablecoins
Stake BTC (look out for SUBFROST protocol token announcements)

The last item here we will discuss in a future article as we approach SUBFROST on mainnet, but it is something of a long saga of development from myself to make this possible.

All of this, for something which seems so simple in ethos: retain exposure to Bitcoin, but use it to earn more Bitcoin, passively. The difficult part is when we would like to do this with zero (or otherwise minimal) protocol risk.

It is my life's work to see such a protocol come to exist.

Follow along:

https://x.com/judoflexchop
https://x.com/bc1SUBFROST

Weapons-grade Metaprotocols on METASHREW

flex — Fri, 29 Nov 2024 21:47:04 +0000

For those who have attempted to extend Bitcoin via the metaprotocol layer, or even just write software to index metaprotocols such as ordinals or BRC-20, you may be aware of the scope of the problem. Most traditional metaprotocols we've seen are easy enough to understand a schema for, as far as the rules to process a single block to update the state of an index.

For ordinals, parsing a block to track the owner of an inscription by a given sequence number, may require program flow such as:

Process each transaction, and if there is a witness stack with an ord-prefixed inscription structure, assign a sequence number and insert into the database
Compute ranges of sats as they are spent onto new outputs, removing their original positions in the set of spendable UTXOs paid into the transaction, and assigning partition ranges instead to UTXOs which are created
Associate sequence numbers for updated owners to their new computed address, based on UTXOs which are able to spend inscriptions associated with a given number

The scope of the ord indexer is greater than this, but this is perhaps one example of a means to design a program which can, when queried, provide a user interface with data about ownership for a given inscription.

These rules are simple enough, but there is much more you have to deal with if you are writing an indexer program.

Some of this work includes:

Retrieving block data from a desired start block height and replaying until the block tip
At tip, wait for new blocks to appear, and when they do, process indexer rules for the new block and update the database
Maintain database connections and implement any query language needed to work against your data store
Detect a reorg and, if needed, roll back state of the index
Provide an API to read from the indexer and render as JSON or any desired format
Retrieve historical state of the index, if we want a view into the dataset at an earlier state
If needed, compute integrity checks of the dataset to ensure there is no corruption and that the state of the index is consistent with others running the same program, without explicitly rebuilding the index from the beginning. Sometimes this is accomplished with merklized tree structures, for example

There is more that one may want to do, but these are some examples of problems that would have to be solved for anyone attempting to index a metaprotocol with software built from scratch.

METASHREW

The METASHREW framework is designed to simplify the task of implementing a metaprotocol, and makes it possible to architect a metaprotocol as intensive as ALKANES. When you build a metaprotocol on METASHREW, instead of building an indexer from scratch, you strictly build a single WASM program, written in any language that is an LLVM frontend, such as Rust.

METASHREW can load a WASM program built with metashrew bindings and run it once for each block in series, automatically detecting reorgs, rolling back state, and processing the corrected block history to correct the state of the index. We choose WASM because it is a portable format for a binary which is platform-independent, where we can deterministically compile the same WASM file to run with identical behavior, with matching compiler software and a version of the source code for the indexer.

With a METASHREW program, it is possible to verify the complete index by simply hashing or otherwise computing a checksum of the WASM program expressing the index, then comparing it to a known value. The same WASM file against the same blockdata will produce the same index in the backend database.

A METASHREW program simply handles the processing of a single block as it seen during a sync of block history from the desired start height. The program is provided a handle to an arbitrary k/v store provided by the runtime. For normal usage of METASHREW we encourage the use of metashrew-keydb provided in the repository, which uses the KeyDB database engineered by the Snapchat team. This database allows us to perform I/O from the indexer directly over a UNIX socket, and it will allow us to do the majority of our work entirely in-memory, if we have a system with a large amount of resources available. This helps with METASHREW programs since we attempt to reuse a local cache of the k/v pairs we store with each block, to avoid context switching out of the WASM program to its most minimal frequency while we rapidly build an index.

METASHREW provides an RPC layer which is designed to be parallelized. View functions are just normal exports from a WASM program, which use the same runtime functions within the indexer environment to access inputs provided with the RPC call.

In ALKANES, we provide a view function to simulate an arbitrary transaction with any mocked inputs of value or cellpack we provide. It is technically possible to execute sandboxed WASM directly within this view function if a contract deployment is simulated against this view function. In this way, a potentially complex view of ALKANES state can be constructed with a view function call, and if the RPC layer is properly multiprocessed on the systems providing metashrew-view service running the alkanes.wasm, we can horizontally scale the rendering model of a user application. This makes it possible, for example, to render graphics or assemble/transform PNG or SVG images, to build more immersive NFTs. In fact, the alkanes-rs repository ships a minimal example of an NFT on ordinals, which we have coined orbitals.

Below is the source code for the most minimal implementation of an orbital:

use alkanes_runtime::{runtime::AlkaneResponder, storage::StoragePointer, token::Token};
use alkanes_support::{parcel::AlkaneTransfer, response::CallResponse, utils::shift};
use anyhow::{anyhow, Result};
use hex_lit::hex;
use metashrew_support::compat::{to_arraybuffer_layout, to_passback_ptr};
use metashrew_support::index_pointer::KeyValuePointer;

#[derive(Default)]
pub struct Orbital(());

impl Token for Orbital {
    fn name(&self) -> String {
        String::from("1x1 Ape (Very f** rare)")
    }
    fn symbol(&self) -> String {
        String::from("APE")
    }
}

impl Orbital {
    pub fn total_supply_pointer(&self) -> StoragePointer {
        StoragePointer::from_keyword("/totalsupply")
    }
    pub fn total_supply(&self) -> u128 {
        self.total_supply_pointer().get_value::<u128>()
    }
    pub fn set_total_supply(&self, v: u128) {
        self.total_supply_pointer().set_value::<u128>(v);
    }
    pub fn observe_initialization(&self) -> Result<()> {
        let mut initialized_pointer = StoragePointer::from_keyword("/initialized");
        if initialized_pointer.get().len() == 0 {
            initialized_pointer.set_value::<u32>(1);
            Ok(())
        } else {
            Err(anyhow!("already initialized"))
        }
    }
    pub fn data(&self) -> Vec<u8> {
        // in this reference implementation, we return a 1x1 PNG
        // NFT data can be anything, however, and it can be computed, if we want a dynamic effect
        (&hex!("89504e470d0a1a0a0000000d494844520000000100000001010300000025db56ca00000003504c5445000000a77a3dda0000000174524e530040e6d8660000000a4944415408d76360000000020001e221bc330000000049454e44ae426082")).to_vec()
    }
}

impl AlkaneResponder for Orbital {
    fn execute(&self) -> CallResponse {
        let context = self.context().unwrap();
        let mut inputs = context.inputs.clone();
        let mut response = CallResponse::forward(&context.incoming_alkanes);
        match shift(&mut inputs).unwrap() {
            0 => {
                self.observe_initialization().unwrap();
                self.set_total_supply(1);
                response.alkanes.0.push(AlkaneTransfer {
                    id: context.myself.clone(),
                    value: 1u128,
                });
            }
            99 => {
                response.data = self.name().into_bytes().to_vec();
            }
            100 => {
                response.data = self.symbol().into_bytes().to_vec();
            }
            101 => {
                response.data = (&self.total_supply().to_le_bytes()).to_vec();
            }
            1000 => response.data = self.data(),
            _ => {
                panic!("unrecognized opcode");
            }
        }
        response
    }
}

#[no_mangle]
pub extern "C" fn __execute() -> i32 {
    let mut response = to_arraybuffer_layout(&Orbital::default().run());
    to_passback_ptr(&mut response)
}

Source: https://github.com/kungfuflex/alkanes-rs/tree/main/crates/alkanes-std-orbital/src/lib.rs

We reserve opcode 1000 for the data view function, which in this case returns a static bytearray, but could in theory be used to pack or construct any dynamic bytearray or imagedata blob, since the view function to simulate transaction messages in ALKANES is run with a very large fuel limit and capable of long running compute.

The METASHREW architecture provides a means to run this kind of compute in an entirely separate process or container, since the data is decoupled from the indexer, and the indexer is decoupled from the view layer.

Compatibility

METASHREW was initially designed as an electrum fork, but I since rewrote it from scratch in order to design an indexer routine that purely operates over the RPC port, to make it more universal for use against Bitcoin compatible interfaces other than Bitcoin itself, including DOGE or DOGE forks.

The libraries for METASHREW are now most up to date as they appear in the Rust monorepo for alkanes-rs, including the pure bindings to METASHREW and also metashrew-support which is an environment-agnostic support library for lower level operations frequently done both in the ALKANES indexer as well as alkane smart contracts themselves.

This includes my own AuxpowBlock structure and implementation, which provides a replacement for the usual rust-bitcoin library, one that can properly parse merge mined blockchains with a slightly modified structure, and requires an extension to the usual Bitcoin block parsing logic.

The implementation of AuxpowBlock provides a convenience method to convert back into a bitcoin::Block structure which drops the Auxpow metadata if we do not need it for our indexer. To index ALKANES, we do not use it.

The result is that the pure Rust implementation of ALKANES, and any other metaprotocol forked from it or built on METASHREW with the support libraries we provide, can run on Bitcoin mainnet, or the many forks we have seen gain some traction in the wild such as Luckycoin, Bellscoin.

And yes, this means we also will have a genesis block for our DOGE L1 smart contracts as well. These genesis constants will likely change as they currently appear in the build system but we will see a canonical ALKANES appear on DOGE L1 on protorunes subprotocol ID 1, sometime following release on mainnet Bitcoin.

I will explain in a future post about why this is a good thing.

ALKANES: Smart Contracts on Bitcoin UTXOs

flex — Thu, 28 Nov 2024 00:08:05 +0000

What is ALKANES?

ALKANES is an application layer built directly on Bitcoin where individuals or teams can deploy smart contracts to Bitcoin L1. ALKANES is one example of a metaprotocol, which is a set of rules and definitions to process block history in a blockchain system, to add additional features and concepts built on its data.

ALKANES is my conception of a metaprotocol project which inherits structures and design patterns from the very popular ordinals and runes metaprotocol projects, authored by Casey Rodarmor. From ordinals, we use the concept of the inscription envelope, but a more minimal variant of it, which we will simply call the witness envelope. We use the witness envelope, at minimum, to deploy *.wasm.gz files to the Bitcoin blockchain, after which they can be transacted against as smart contract programs, using any combination of inputs of value, data, and the context of the transaction itself.

With the *.wasm.gz file format of ALKANES smart contracts, we can build smart contracts in a language like Rust, and build contracts that are very much within the size constraints we must work within, when deploying to something like the Bitcoin blockchain. In fact, as an experiment, I forked the noir VM to a slightly modified version that would play nice with the alkanes build system, and I used it to do a minimal build of an alkane smart contract capable of verifying a proof for a zk-circuit compiles with the AZTEC noir project. The final binary filesize was 550kb, but when gzipped to a *.wasm.gz, was merely 150kb.

This means there is a world of possibilities for what we can accomplish on Bitcoin as it exists today.

For its regular operation, ALKANES inherits from the runes metaprotocol as well as ordinals. We do in fact use the Runestone structure as it is defined in the runes specification, but in a way that is harmless to indexers tracking the state of runes, and also, in a way where it is possible to compose transfers of runes and alkanes as part of a swap transaction, which has the correct effect on indexers tracking the state of either metaprotocol.

Inheriting from these two metaprotocols means we have a lot of open source software in the ecosystem already that we can tap into. I ported modified versions of @magiceden-oss/runestone-lib and also micro-ordinals from the desk of the prolific https://github.com/paulmillr, into the alkanes TypeScript repository here:

https://github.com/kungfuflex/alkanes

This library is usable for constructing transactions that can both deploy and transact against alkanes smart contracts, and an example of a complete program flow can be found here:

https://github.com/kungfuflex/alkanes/tree/main/integration/genesis.spec.ts

In this repository I have also written a docker-compose.yaml which provides a full regtest environment and supplementary database, usable to run the integration test mentioned above, but also to build applications! Indeed, it is possible to build applications that are even more expressive than what we have seen in traditional DeFi systems, using the combined power of LLVM and the paritytech wasmi interpreter / WebAssembly runtime, entirely on native Bitcoin L1.

What can an ALKANE do?

An alkane smart contract is very much a Bitcoin native construct. First and foremost, it is always a token, whether or not it actually does anything with its own balance sheet. It can hold a balance sheet of itself as well as other alkanes, the same that a UTXO can hold alkanes or runes. This idea actually resonated with me when Andre Cronje explained to me his thesis on DeFi many years ago, applied to his vision for YFI.

In simple terms: "make every contract an ERC20."

This happens to provide a consistent model for interoperability that helped DeFi take form, and I wanted this to exist as a primitive within ALKANES, bearing the improvement that a transfer of assets always invokes the code of the recipient, so we can always do everything in one transaction in the UX we build.

What else?

An alkane can:

Check balances of alkanes
Read/write to storage for the alkane
Check how much fuel is left to execute code
Access the transaction bytes of the Bitcoin transaction executing the alkane
- If it is present, read one or more witness envelope structures present with the transaction, when we need to send a transaction with large payloads such as a zk proof, merkle proof, or quantum safe signature
- Compute the address that the output will be spent to, or otherwise check the Bitcoin vmscript on the output that the output of some alkanes action will end up on, if we want to check for covenants, enforce conditions on spending, or other novel uses of OP_CAT even. Building on UTXOs mean we have the benefit of being able to extend Bitcoin scripting as we normally use it, and get the benefit of new opcodes as they occur, if they do.
Access the entire block bytes which the current transaction is running within
- Programmatically determine who the miner of the current block is (we may want to bribe this individual in our transaction, perhaps)
Act upon the transfer of value sent with the transaction, in terms of the combined balance sheet of alkanes spent into the action
Access the ID for the alkane calling the current alkane, if they call each other
Access the ID for the alkane executing code
Call another alkane
Delegate to another alkane (upgradeability, libraries)
Call another alkane but prevent side effects
Get the next sequence number, usable also if an alkane is created by an alkane in a transaction and its ID should be computed

I tried not to get too much fancier than this, since everything else should be doable in userspace.

Indexer

The indexer for ALKANES itself actually builds to WASM. So, it is true that the WASM VM for an alkane, is itself, built to WASM. The reason it was possible to build a metaprotocol that was deterministic and which meets the strict integrity requirements of a metaprotocol indexer built on top of a blockchain consensus model, is because we have something like metashrew, which is a generic metaprotocol indexer stack capable of running WASM programs expressing a metaprotocol.

https://github.com/sandshrewmetaprotocols

Metashrew is itself an extensive project and warrants a separate article to properly introduce, but the easy way to put it is that it is a software stack that reduces the problem of architecting a complex metaprotocol, by a considerable degree.

The ALKANES indexer program is a pure Rust project built with bindings to the metashrew runtime. To verify that the version of the ALKANES indexer, and therefore the index it builds, is consistent with what is expected, it is simple enough to just compare the SHA256 hash of the WASM binary for alkanes.wasm to a known value. I have actually had to do this quite a few times for easy sanity checks while debugging in the indexer WASM. Very convenient.

The metashrew software comes with metashrew-view, which is a minimal JSON-RPC service that will execute the same WASM for the indexer but in a way where there cannot be side effects of calling function. The process can be parallelized and provides a way you can multiprocess potentially heavy compute to render the state or a simulated effect of the indexer. In ALKANES we actually export a simulate function that allows you to run any protocol message or transaction against an alkane WASM, and this functions similar to what we are used to from Ethereum and refer to as view functions, but it is more comprehensive. You can simulate and compute fuel costs of a protocol message, mock any inputs desired as the balance sheet of assets you are spending into the transaction, get the list of expected alkanes that will be output from the transaction, see the list of storage updates as a result of the transaction, and of course, do all of it as if we are at an earlier block height than the current tip, if we want to see historical state.

What's next?

Well, it is true that people are excited already. We have projects deploying stablecoins, AMM systems, explorers, and of course we will have wallet support within OYL coming, so many of the pieces are already there or otherwise are staged to appear.

Join us in the SANDSHREWサンド Discord where we discuss ALKANES and the greater ecosystem of metaprotocols we will come to know, as we enter the next chapter of Bitcoin.

https://discord.gg/wSrZtSrnrf