Advanced Developer Tutorial on Address Lookup Tables

Since the early times when data begun to be used to compute values, there has always been a need to store complex data in such a way that they would always be easy to access. In ancient times this was accomplished with sine tables and a 98-column multiplication, advances in computing technology has impacted the way data is stored and retrieved.
In the early years of computer technology, processing input/output was particularly slow and would consume more resource and time. To solve this problem, it made sense to reduce significantly the process of reading operations and operations data. A very efficient method that was introduced to solve this problem was Caching (which involves storing data so that it can be available for future request at faster speeds due to an earlier computation of the data stored elsewhere).
However, while systemwide caching tries to automate the process of fetching commonly occurring data items, to enable a much faster mechanism for obtaining data items that possess a sort of immutability, lookup tables were introduced.

What is a Lookup Table

As if defined in computer science, [1]

a lookup table(LUT) is an array that replaces runtime computation with a simpler array indexing operation. This process of utilizing array indexing to compute the running phase of a program is also known as direct addressing. It is much faster and less expensive since data is retrieved from memory using indexes as reference compared to the processing input/output approach. The tables may be pre-calculated and stored in static program storage, calculated (or "pre-fetched") as part of a program's initialization phase, or even stored in hardware in application-specific platforms.

A lookup table is more efficient in terms of lookup operation and has a guaranteed O(1) time complexity. However, it is not possible for two entities to have the same lookup key k, and as the size of the datasets grow increasingly larger, in the magnitude of considering all forms of allottable data, it becomes less practical to stored a LUT in memory.

Address Lookup Table (ALTs)

As defined on the Solana documentation, [2]

An address Lookup Tables, commonly referred to as "lookup tables" or "ALTs" for short, allow developers to create a collection of related addresses to efficiently load more addresses in a single transaction. Since each transaction on the Solana blockchain requires a listing of every address that is interacted with as part of the transaction; without ALTs, this listing would effectively be capped at 32 addresses per transaction, but with ALTs, it would be effectively raise the addresses per transaction to 256

ALTs are part of additional functionalities that were introduced in the versioned transaction format which is quite similar to the legacy transaction format with just slight differences in libraries used.

How to Create An Address Lookup Table(ALTs)

This tutorial is going to show you how you can implement ALTs on your projects and help you see the benefits and advantages they offer.

What We Will Implement in This Tutorial

In this Tutorial, we will:

• Create and execute a Version 0 (V0) Transaction
• Create and Populate an Address Lookup Table
• Compare the transaction size of two nearly identical transactions where one would implement a lookup table and the other would not

Perquisites:

• Node.js
• Intermediate Experience with Typescript/JavaScript
• ts-node
• basic or solid experience in running Solana transactions
• all necessary imports including classes from Solana Web3 library
• create a wallet and fund it with some devnet SOL
• set up API endpoint to connect with the Solana network

Your setup Environment should look similar to this:

Steps to Creating ALTs

Executing Versioned Transaction V(0)

To use Lookup Tables, you must execute a versioned transactions.

We can do this by Let's start by creating a new function, createAndSendV0Tx, that accepts an array of TransactionInstructions, txInstructions:

async function createAndSendV0Tx(txInstructions: TransactionInstruction[]) {
// Step 1 - Fetch Latest Blockhash   
let latestBlockhash =await SOLANA_CONNECTION.getLatestBlockhash('finalized');    console.log("   ✅ - Fetched latest blockhash. Last valid height:", latestBlockhash.lastValidBlockHeight)
;    

// Step 2 - Generate Transaction Message    
const messageV0 = new TransactionMessage({ payerKey: SIGNER_WALLET.publicKey,  recentBlockhash: latestBlockhash.blockhash, instructions: txInstructions    }).compileToV0Message();    console.log("   ✅ - Compiled transaction message");    
const transaction = new VersionedTransaction(messageV0);    
// Step 3 - Sign your transaction with the required `Signers`    transaction.sign([SIGNER_WALLET]);    console.log("   ✅ - Transaction Signed");    // Step 4 - Send our v0 transaction to the cluster    const txid = await SOLANA_CONNECTION.sendTransaction(transaction, { maxRetries: 5 });    console.log("   ✅ - Transaction sent to network");    
// Step 5 - Confirm Transaction     
const confirmation = await SOLANA_CONNECTION.confirmTransaction({signature: txid,        blockhash: latestBlockhash.blockhash, lastValidBlockHeight: latestBlockhash.lastValidBlockHeight    });  if (confirmation.value.err) { throw new Error("   ❌ - Transaction not confirmed.") }    console.log('🎉 Transaction succesfully confirmed!', '\n', `https://explorer.solana.com/tx/${txid}?cluster=devnet`);}

Let's walk through our code:

Step 1: We fetch the latest blockhash from the network. Note: We pass the parameter, 'finalized', to make sure the block does not belong to a dropped fork.
Step 2: Using our txInstructions parameter and the latestBlockhash, we create a new MessageV0 by building a Message and executing the .compileToV0Message() method.
Step 3: We sign the transaction with an array of signers. In this case, it is just our SIGNER_WALLET.
Step 4: We send the transaction to the cluster using sendTransaction, which will return a transaction signature/id.
Step 5: We wait for the cluster to confirm the transaction has succeeded. If it succeeds, we log our explorer URL; otherwise, we throw an error.

Once successful with the above, we now move on to create an ALT

Create an Address Lookup Table

Create a new async function, createLookupTable, that will build our transaction instruction and invoke createAndSendV0Tx:

async function createLookupTable() {
    // Step 1 - Get a lookup table address and create lookup table instruction
    const [lookupTableInst, lookupTableAddress] =
        AddressLookupTableProgram.createLookupTable({
            authority: SIGNER_WALLET.publicKey,
            payer: SIGNER_WALLET.publicKey,
            recentSlot: await SOLANA_CONNECTION.getSlot(),
        });

    // Step 2 - Log Lookup Table Address
    console.log("Lookup Table Address:", lookupTableAddress.toBase58());

    // Step 3 - Generate a transaction and send it to the network
    createAndSendV0Tx([lookupTableInst]);
}

Breaking down our code:

Step 1: We create two variables, lookupTableInst and lookupTableAddress, by destructuring the results of the createLookupTable method. This method returns the public key for the table once created and a TransactionInstruction that can be passed into our createAndSendV0Tx function.
Step 2: We log the table's address
Step 3: Finally, we call createAndSendV0Txby passing lookupTableInst inside of an array to match our type requirements.

Create an empty address table by running:

createLookupTable();

To run the transaction, enter your terminal and execute:

ts-node app.ts

After the transaction has completed, you should receive a URL to the transaction page on Solana Explorer, like in the image below:

The lookup table account address is: 3uBhgRWPTPLfvfqxi4M9eVZC8nS1kDG9XPkdHKgG69nw

Next steps:

1. Remove the call to createLookupTable()
2. add the table lookup address from your console to the PublicKey declaration like so :

const LOOKUP_TABLE_ADDRESS = new PublicKey("YOUR_TABLE_ADDRESS_HERE"); 
// e.g const LOOKUP_TABLE_ADDRESS = new PublicKey("3uBhgRWPTPLfvfqxi4M9eVZC8nS1kDG9XPkdHKgG69nw");

So what's going on with the extendLookupTable method:

1. We pass our SIGNER_WALLET to pay the transaction fees and any additional rent incurred.
2. We define our update authority - in our case, we set that as the SIGNER_WALLET in our table creation step above.
3. We pass in the lookup table account address (which we defined as LOOKUP_TABLE_ADDRESS).
4. We pass an array of addresses into our lookup table. We will pass in a few random public keys, but you can pass in any public key that you like! The Program's "compression" supports storing up to 256 addresses in a single lookup table!
5. log a link to our lookup table entries for easy access after the transaction is complete. Finally, we pass TransactionInstruction into createAndSendV0Tx to generate a transaction and send it to the network!

Call the new function:

addAddressesToTable();

Now run the in the terminal using the previous ts-node app.tscommand. Check the returned URL to see your transactions and your lookup table entries on the Solana Explorer. On the lookup table entries, there will be a list of all the stored public keys.

By now perhaps you are asking, why is this better? Well, The power of ALTs in transaction lies is in the efficiency of the transaction! By effectively compressing a 32-byte address to a 1-byte index value
This is great for speed as well as space efficiency as you can push in more transactions without bothering about space. By eliminating the need for long and complex iterations in cases of numerous addresses, which leads to error-resistance as ALTs can be engineered to prevent duplicate addresses.
Also this is cheaper; by reducing the number and size of the instructions in a transaction, thereby making fees required to develop on the blockchain way more cost-effective.
From transaction caching operations, scalability, speed, cost, the advantages of the address lookup table cannot be overemphasized, the benefits are numerous!
of course they are not the best when it comes to dealing with a dataset of unpredictable large number of possible keys, and could exceed the number of keys that are actually stored. For this sort of cases, hash tables are more efficient. Read more about hash tables here

While most use cases of ALTs revolve around transactional caching, decentralized applications employ their power to store multiple address for easy access at O(1) time; within a single transaction, you can load a large number of addresses

A very practical use case for ALTs can be found in asset management dApps where multiple wallets are being monitored; without Solana ALTs, it would be a resource consuming process.

For further reference, You can find another simple example to build on the example we have shown already Github

Conclusion

We have learned the following points in this terse tutorial:

• In blockchain transactions, the magnitude of addresses used in multi-level transactions can easily scale in magnitude of millions of addresses; hence requiring an efficient means of retrieving transaction data in an O(1) time
• Lookup tables are best for this as they utilize a concept known as "direct addressing" which utilizes array indexing to compute the running phase of a program.
• By utilizing Address Lookup Tables, blockchain developers can create a collection of related addresses to efficiently load more addresses in a single transaction; address tables can store up to 256 addresses each. This can however be a limitation in special cases. In addition to stored addresses, address table accounts also tracks various metadata.