Solana DEV #01: Fetching onchain data in a right way

Working with Solana decentralized app from a client side can be a core part of the Solana app development. Before being comfortable with the process, I thought smart contract programming is the one and only concern that we should thoroughly care about. However, similar to traditional website development, data fetching on client side is impactful to the experience delivered to user. In this article, I will list out key notes of data fetching.

Get filtered program accounts

First and foremost best practice is using getProgramAccounts with a filter. This getProgramAccounts method in @solana/web3.js package is extremely useful for fetching all accounts of a provided program ID. However, as in a smart contract, there would be multiple type of accounts, filtering is required for best performance.

/**
 * A filter object for getProgramAccounts
 */
export type GetProgramAccountsFilter = MemcmpFilter | DataSizeFilter;

Now we play with the source code a bit. There are two filtering types that we can reference MemcmpFilter and DataSizeFilter.

DataSizeFilter

/**
 * Data size comparison filter for getProgramAccounts
 */
export type DataSizeFilter = {
  /** Size of data for program account data length comparison */
  dataSize: number;
};

DataSizeFilter is used to filter accounts of a program matched with a declared data size. Simple right? This filtering type is useful for accounts with fixed data size. With accounts contain dynamic size fields such as vector or str.

MemcmpFilter

/**
 * Memory comparison filter for getProgramAccounts
 */
export type MemcmpFilter = {
  memcmp: {
    /** offset into program account data to start comparison */
    offset: number;
    /** data to match, as base-58 encoded string and limited to less than 129 bytes */
    bytes: string;
  };
};

Between the two filtering types, I find this approach is used more as it allows you to filter accounts in a more dynamic and selective way. MemcmpFilter or Memory comparison filter has two fields: offset and bytes. Every account data is a set of bytes, you can use offset to choose specific data field to compare and bytes is a the later part of a comparison.

For example, if I want to get all program accounts with a specific account discriminator, I can set the offset as 0 and get the bytes of the compared account discriminator. This is a trick to get specific type program accounts without using Anchor (as the framework already handle this process under the hood).

I usually use this filtering types to filter out which accounts that a specific wallet owned. To do this, in the Rust program, you must store the wallet address of the owner in the account. For example, I have this account data (using Anchor):

#[account]
pub struct WalletAddressContainer {
  pub address: PubKey
}

To fetch every WalletAddressContainer accounts of a specific wallet address, I would do

// using Anchor
await program.account.flow.all([
 {
  memcmp: {
    offset: 8,
    bytes: publicKey.toBase58(),
  },
 }
]);

// without Anchor
await connection.getProgramAccounts(
 programID,
 {
  ...,
  filters: [
   {
    memcmp: {
     offset: 0,
     bytes: /** Account discriminator bytes of WalletAddressContainer **/,
    }
   },
   {
    memcmp: {
     offset: 8,
     bytes: publicKey.toBase58()
    }
   }
  ]
 }
);

In an Anchor way, as mentioned, Anchor framework handles comparing account discriminator under the hood, so we don't have to compare it again. In that way, we only need one memcmp filter to compare the public key of the wallet but the offset is still have to be 8 because the first 8 bytes are for account discriminator.

On the other hand, getProgramAccounts without Anchor requires two comparison. First the account discriminator and second the wallet address.