Satoshi S. for Web Dev Path

Posted on Apr 6

My Experience with Solana

#solana #python #mlh #data

Intro

I’ll start with an update on my current work.

Last November, I began working with JB Music Therapy to build a music wellness application. We recently completed a large user testing phase with over 250 participants, including music therapy students. Now, we’re migrating the web application to Flutter in preparation for releasing it on the Google Play Store and Apple App Store.

Since I don’t yet have production-level mobile development experience, I’ve been actively learning through a Udemy course while working on the migration. It’s been a challenging but rewarding process.

As a contract worker, I have a flexible schedule. To take advantage of that, I decided to apply to the MLH Fellowship again. I’ve participated in the program three times before, and each time I’ve gained valuable skills and experience.

This time, I was accepted into the Web3 track and assigned to work with the Solana Foundation team. In this post, I’ll share my experience working with Solana.

If you're interested in my previous experiences in the program, please check the posts below.

Project We Were Assigned To

Usually, MLH Fellowship participants contribute to the existing large-scale open-source projects under the guidance of core maintainers, but this time was a bit different. Our team was tasked with building a system to aggregate stablecoin metrics from multiple data providers and present them in a unified dashboard for the Solana Developer Website.

Why is this important? Because metrics can vary depending on the data provider. Differences in calculation methods, definitions, and data sources often lead to inconsistencies. Having a single place to visually compare these metrics makes it easier to understand and validate the data.

To support this, we are building a data pipeline on Databricks that pulls data from five major cryptocurrency analytics platforms: Allium, Artemis, Blockworks, DefiLlama, and Dune.

We are in the process of open-sourcing the data retrieval repository so that other developers can easily add new data providers or extend the available metrics in the future.

Explore the API

As a starting point, we decided to retrieve four core metrics.

metrics = [
  {
    'name': 'Supply',
    'unit': 'USD',
    'description': 'Total circulating supply of stablecoins on Solana, denominated in USD.'
  },
  {
    'name': 'Transfer Volume',
    'unit': 'USD',
    'description': 'Total transfer volume of stablecoins on Solana, denominated in USD.'
  },
  {
    'name': 'Transfer Count',
    'unit': 'Count',
    'description': 'Total number of stablecoin transfer transactions on Solana'
  },
  {
    'name': 'Active Addresses',
    'unit': 'Count',
    'description': 'Number of unique addresses interacting with stablecoins on Solana'
  }
]

We explored each API using notebooks to retrieve these metrics and understand their data structures.

During this process, we encountered several challenges, such as handling paginated responses, working with SQL-based APIs and adapting to different data schemas across providers.

Ingestion Pipeline

After exploring data provider APIs, we moved on to building the ingestion pipeline. We set up a database on Databricks and stored the data in a consistent format. During this process, we also began structuring the codebase with the goal of open-sourcing it in the future.

Dashboard

Databricks provides built-in dashboarding capabilities, which we used to visualize the aggregated data. I leveraged the AI Agent to generate plots quickly, allowing us to compare metrics across providers. Databricks also provides embeddable HTML components, which we plan to integrate into the final Solana developer page.

Repository Structure Proposal

We are now halfway through the Fellowship and have started organizing our code from Databricks into a structured GitHub repository.

solana-stablecoin-benchmark/
  providers/        # contributors add providers here
  normalizers/      # shared transformation logic
  pipelines/        # ingestion + normalization orchestration
  db/               # schema definitions
  config/           # config files (logging, env templates)
  tests/            # validation and correctness
  notebooks/        # optional exploration (Databricks / local analysis)
  main.py           # pipeline entry point
  requirements.txt  # dependencies
  .env.example      # API keys template
  README.md

This structure is designed to make the project easy to extend, allowing contributors to add new data providers and metrics with minimal friction.

Conclusion

Throughout the project, effective communication has been just as important as technical implementation. Since our team is distributed across multiple time zones, we rely heavily on asynchronous communication.

To support this, I've focused on clearly documenting my work so that teammates can easily understand progress and continue development without blockers. This experience has reinforced how important well-written documentation is for team productivity.

This project has been a great opportunity to deepen my experience in data engineering and collaborate in a distributed team environment.

Note

I plan to update this post at the end of the program with the final results of our work. I hope you can check it out on GitHub and the Solana website.

DEV Community