Synternet Data Layer

• Introduction: Synternet and the Data Layer
  • Synternet: Powering Data Infrastructure for Web3
  • The Data Layer: The Leading Protocol for Real-Time Data
• Prerequisites
• Setting Up the Environment
• Exploring the Data Layer with the Synternet Golang SDK
• Conclusion

Introduction

As the blockchain landscape explodes with new projects and protocols, a major challenge emerges: data exchange and interoperability.

Traditional blockchains often operate in silos, built with their own unique structures and protocols. This isolation makes it difficult for them to seamlessly communicate and share information.

Imagine a world where each bank operates on its own isolated financial network, unable to exchange funds or verify transactions with other banks. That's the current state of many blockchains – a fragmented ecosystem hindering the full potential of a connected web3 world.

Now, imagine a layer exists where real-time blockchain data can be exchanged securely and efficiently, and this is what Synternet brings to Web3.

Synternet: Powering Data Infrastructure for Web3

Synternet tackles this data isolation problem by offering an interoperable data infrastructure across all major chains. It functions as a bridge between blockchains, enabling them to exchange data securely and efficiently. This interoperable data layer empowers developers to build applications that leverage data from various blockchains, fostering a more interconnected web3 ecosystem.

The Data Layer: The Leading Protocol for Real-Time Data

At the heart of Synternet lies the Data Layer, its core protocol acting as a bridge between blockchains. Imagine a universal translator for blockchain data – that's the essence of the Data Layer. It empowers developers with instant access to any cross-chain data, fostering a truly interconnected web3 ecosystem.

This functionality is achieved through a publish-subscribe (pub-sub) framework where data providers can continuously stream live data (like current prices or transaction details) to applications and smart contracts. This approach eliminates the need for traditional request-reply oracles, which can be computationally expensive and introduce centralization risks. By contrast, the Data Layer offers real-time, ultra-low-cost data delivery without any middlemen, promoting a more efficient and secure web3 environment.

In this article, we will explore the data layer using the Synternet Go SDK.

Prerequisites

Before we dive in, ensure you have the following tools and prerequisites in place:

• Go: Follow the installation guide.
• Nats: Set up and start the NATS server using this guide. For Windows users, I recommend using chocolatey or Go for installation.
• Nats CLI: Install Nats CLI to help interact with subjects from the terminal. Use this guide
• Amber Testnet Tokens: To help interact with the Data Layer. Geyt some from the faucet.
• Create a Synternet project to get Access Token. Use this guide
• Subscribe to a Stream

Setting Up the Environment

Now that we've gathered our tools, it's time to set up our development environment. Here's a step-by-step guide:

• Create a new file directory and enable dependency tracking for your code.

mkdir synternet

cd synternet

go mod init synternet

code .

• Run the following command to install all the necessary packages

go get github.com/SyntropyNet/pubsub-go/pubsub 
go get github.com/joho/godotenv 
go get github.com/nats-io/nats.go/

• Create a new go file and import the following packages.

pacakage main

import (
    "context"
    "fmt"
    "log"
    "os"
    "os/signal"
    "syscall"


    "github.com/SyntropyNet/pubsub-go/pubsub"
    "github.com/joho/godotenv"
    "github.com/nats-io/nats.go"
)

• To keep sensitive information like your access key, create a .env file in your project directory and store your keys there in the format below:

ACCESS_KEY="THISISMYACCESSKEY"

• Create these as global variables as we would need them all through.

// Configuration variables
const (
    natsUrl = "nats://127.0.0.1:4222"  // Default NATS server address
    subject = "staking.osmosis.mempool"  // The subject we want to subscribe to
)

// Access token retrieved from environment variable
var  accessToken  string // Would be initialized in the main function

Now we have all we need, let's dive in.

Exploring the Data Layer with the Synternet Golang SDK

We would get started by creating a function PrintData which would serve as a message handler to process and log data about the received message.

func  PrintData(ctx  context.Context, service *pubsub.NatsService, data []byte) error {
log.Println("Received message on subject:", subject)
fmt.Println(string(data)) // Print the message data as a string
return  nil
}

It takes in 3 arguments:

• ctx: This argument is of type context.Context. It provides information about the context of the function call, including cancellation signals.
• service: This argument is a pointer to a pubsub.NatsService object. This object represents the connection to the NATS server and provides methods for message handling.
• data: This argument is a slice of bytes ([]byte). It contains the actual data received from the subscribed subject. Proceeding to our main function, we would start by initializing our accessToken from our .env file.

func main() {
    // Load environment variables from .env
    err := godotenv.Load()
    if  err != nil {
    log.Fatal("Error loading .env file: ", err)
    }

    // Read access token from the environment variable "ACCESS_TOKEN"
    accessToken = os.Getenv("ACCESS_TOKEN")
    fmt.Println("Access Token:", accessToken) // For debugging, remove for security issues
}

Running go run main.go would have our access token printed to the console.
We would go ahead to use this access token to generate a JWT token for authorization with the NATS server.

jwt, err := pubsub.CreateAppJwt(accessToken)
if  err != nil {
    log.Println("Error creating JWT token:", err)
} else {
    fmt.Println("Generated JWT token:", jwt) // Log the generated JWT token (for debugging)
}

This would allow us to establish a connection with the NATS server for subscribing to messages.

// NATS connection options with User JWT and Seed (access token) for authentication
opts := []nats.Option{
nats.UserJWTAndSeed(jwt, accessToken),
}

// Connect to the NATS server using the configured URL and options
service := pubsub.MustConnect(pubsub.Config{
URI: natsUrl,
Opts: opts,
})
log.Println("Connected to NATS server successfully.")

// Create a context with cancellation functionality
ctx, cancel := context.WithCancel(context.Background())
defer  cancel() // Ensure cancellation on exit

// Informational message
fmt.Println("Waiting for messages...")

Run the command to confirm your code works just fine: go run main.go

You should get something like this as an output:

Access Token: SAAHDFQNIJ5S5DENJPUJAVGRHSQZRYZLT7NJVBBAOWYBSP6AWNDZZZ7KVU
Generated JWT token: eyJhbGciOiJlZDI1NTE5LW5rZXkiLCJ0eXAiOiJKV1QifQ.eyJqdGkiOiIxNzE1MzgwNDE1NjIyMzAwNDE1NDgzNjIxIiwiaWF0IjoxNzE1MzgwNDE1LCJpc3MiOiJBQ0tVNEk2VjI2QUFUQTQ2U1FRT1dFUjZCTVBCMlk1RDZUWVc2Q0QyMk00Tk1IQUlGSVJPNUczSyIsIm5hbWUiOiJkZXZlbG9wZXIiLCJzdWIiOiJBQ0tVNEk2VjI2QUFUQTQ2U1FRT1dFUjZCTVBCMlk1RDZUWVc2Q0QyMk00Tk1IQUlGSVJPNUczSyIsIm5hdHMiOnsiZGF0YSI6LTEsInBheWxvYWQiOi0xLCJwdWIiOnt9LCJzdWIiOnt9LCJzdWJzIjotMSwidHlwZSI6InVzZXIiLCJ2ZXJzaW9uIjoyfX0.XFwuojtVox2MxW0vAh5hitjnIymLqR2fIAZZR-1zFxI_QFlhPjyRB7L_WA6SgBIRY0IM8O5HA61dRY_4WoYNAg
2024/05/10 23:33:35 Connected to NATS server successfully.
Waiting for messages...

Next, we subscribe to the subject and create a channel to receive signals from the system.

// Subscribe to the specified subject using a handler function
service.AddHandler(subject, func(data []byte) error {
    fmt.Println("Received message data:")
    return  PrintData(ctx, service, data) // Process data using PrintData function
})

signalChan := make(chan  os.Signal, 1)
signal.Notify(signalChan, syscall.SIGINT, syscall.SIGTERM)
go  func() {
    <-signalChan
    cancel() // Cancel the context on receiving SIGINT or SIGTERM
}()

// Start serving messages and handle them using the registered handler
service.Serve(ctx)
fmt.Println("Exiting application...")

By following this guide, you'll be equipped to leverage the Synternet Data Layer and receive real-time blockchain data streams using Go. To get a sense of the data formats and potential applications, explore the Synternet Showcase: https://showcase.synternet.com/

You can find the full code for this tutorial here. Feel free to make contributions.

Conclusion

With this guide, you have learned:

• The Data Interoperability Problem and how Synternet solves it
• Synternet Data Layer
• Building with the Golang SDK

This is just the beginning. As you delve deeper into the Synternet network, consider exploring the following resources:

• The Synter Data Layer
• The NATS Docs
• Start building with Synternet
• Synternet Blog
• Synternet Builder's Community

Comments

[Quadri Omoshalewa Olabisi]

Thank you tosynthegeek! Your ability to thoroughly explain yet simplify your write ups is really amazing!!!
I understood every bit of it.
This is yet another wonderful article!