DEV Community

Shaikh Sadi
Shaikh Sadi

Posted on

Managing Large DataList In Solana

Image description

This is a continuation of my solana journey. Here I will share my experience of using solana.

Everything is stored in Account

In solana everything is stored in account. So a program is stored in account , your wallet data is stored in account, any user data who is interacting with any program that is also stored in account. As of now i used anchor for building solana programs so I will try to explain in that perspective.

Default Way

The most easiest and low maintenance way is to store data in a vec. In this way your account could go up to 32kb. But there is a catch, I used to calculate the space and count how many items i can push into the vec before hitting that 32kb stack size. But unfortunately i was not able to reach that count, it was failing almost in the half way with Memory allocation failed error.
After many finding i came to know this is due to rust, rust doesn't allocate memory on every push, Not sure but I think they try to allocate double of the current size once you reach the current size. Although there is workarounds around for this.

Zero Copy Account

With zero-copy account you can reach 10mb size, this is the maximum account size that solana allows. The biggest issue is you need to pre-define array length , yes array. You can't use vector in zero-copy account. As a result when creating account user will have to pay much more even though they won't use all the space.

Splitting Account

If other two options are not suitable for you , you may consider this way. The idea is you will store every element in its own account(I am currently experimenting with this approach). On positive side , it can grow almost infinitely , only pay for one account at once.
But management can get hard in this approach, like you will have to decide how you will generate the seed for each element, you wanna reuse the space of previously allocated unused account, etc etc....

Third one is more of a structural solution. But you can choose any of them based on your necessity and requirements. I am not an expert , so I would really love to hear any correction or idea. Thank You.

Top comments (0)