Step-by-Step Guide How to Store your data on IPFS with Spheron Browser SDK

Introduction

When creating a dApp, we usually use a blockchain like Ethereum. Imagine you are building a decentralized social media platform on Ethereum. Users can share images, videos, and text. But how will you store all this content in a decentralized way? Usually, files are kept on a central server like AWS, GCP, or Azure, but that can be costly and inefficient.

To solve this, we use IPFS, a decentralized and distributed protocol to store and share files in a peer-to-peer network. Instead of relying on one server, IPFS spreads files across many computers connected to the network. This makes files more available, secure, and reliable.

In this blog post, we'll explore how you can harness the power of IPFS with Spheron's Browser Upload SDK to store data in a decentralized, secure, and cost-effective manner.

Why Use Spheron's Browser Upload SDK?

Spheron's Browser Upload SDK is a tool that makes it easy for developers to upload files directly from a web browser to services like IPFS, Filecoin, or Arweave. It handles all the complexities of interacting with the IPFS network, so you can focus on building your application without worrying about the details. The SDK also offers features like encryption, access control, and streaming, making it a powerful tool for creating modern web applications.

One of the main advantages of using browser upload SDK is that you can avoid writing complex code for file uploading, such as multi-chunking and parallelization. Another advantage is that it takes care of storage scalability and redundancy, so you don't have to worry about managing lots of server space or developing a complicated mechanism to load balancing on your server instances.

Prerequisites

Before we begin, make sure you do the following:

• Install Node.js and npm (Node Package Manager) on your machine.

• Create a Spheron account from here.

• Before creating an access token for authentication, new users need to sync their GitHub or GitLab account to give Spheron access.

Step 1: Creating an Access Token

To generate and manage Access Tokens for your Spheron dashboard app, follow these steps:

1. Go to your profile icon in your Spheron dashboard app and click on it.

2. Click on settings from the image above to your settings page and navigate to the Tokens in your Account Settings.

3. Click Create to open a new Access Token.

4. Enter a descriptive name, choose the scope from the list of Organizations in the drop-down menu, select an expiration date for the token, and click create token.

By following these steps, you'll have successfully created an access token. Creating an access token allows you to authenticate and authorize your applications to interact with the Spheron API and utilize its features and functionalities. The scope ensures that only your specified Organization can use an Access Token. Once you've created an Access Token, securely store the value, as it will not be shown again.

Note: When creating the access token, choosing the "web app" type in the dashboard is recommended. This token can be used with various Spheron SDKs, such as the Storage SDK and the Compute SDK.

Step 2: Installing the Spheron Browser SDK

To begin, You will need to install the Spheron Browser SDK. If you're working with npm, use the following command in your terminal to install the SDK:

npm i @spheron/browser-upload

For yarn, run the following command instead:

yarn add @spheron/browser-upload

To use the Browser SDK for file uploads, you'll need a web server to provide a uploadToken for data upload and a web app with the @spheron/browser-upload module to enable direct browser-based file uploads.

Step 3: Setting Up the Server

You need to set up an express server with an endpoint GET: /initiate-upload that will be used by the client project to get the uploadToken and to use it to upload data. In the example, we have created an /initiate-upload endpoint. The createSingleUploadToken method from the @spheron/storage package generates the uploadToken. All the logic is in index.js.

To start the project:

1. Execute npm install

2. Create a .env file in the root and SPHERON_TOKEN variable with the value of your Spheron API Token.

3. Execute npm start
   

// Import the necessary modules from the @spheron/browser-upload package
import { SpheronClient, ProtocolEnum } from "@spheron/storage";

// ...

// Set up an endpoint on the server to fetch the uploadToken
app.get("/initiate-upload", async (req, res, next) => {
  try {
    const bucketName = "example-browser-upload";
    const protocol = ProtocolEnum.IPFS; // desired protocol
    const token = process.env.SPHERON_TOKEN; 

    // Create a new SpheronClient instance with the access token
    const client = new SpheronClient({ token });

    // Generate the uploadToken using createSingleUploadToken method
    const { uploadToken } = await client.createSingleUploadToken({
      name: bucketName,
      protocol,
    });

    // Respond with the uploadToken in JSON format
    res.status(200).json({
      uploadToken,
    });
  } catch (error) {
    console.error(error); // Handle any errors that may occur
    next(error);
  }
});

Step 4: Web App Setup

You must create a React web app to upload files directly from the browser using a package called @spheron/browser-upload. The app will first ask the server (Backend Service) for uploadToken to enable secure file uploads. Once it receives the token from the server, it will upload the files. For more reference, check out this example .

To start the project, follow these steps:

1. Run npm install in your project directory to install the necessary dependencies.

2. Then, run npm start to start the web app.

The browser upload provides three methods:

1. Upload your files

To upload files from the browser, you must first request the uploadToken from the server. Then, you can use the upload method from @spheron/browser-upload to upload your files.

// Import the necessary modules from @spheron/browser-upload package
import { upload } from "@spheron/browser-upload";

// ...

// Fetch the temporary access token from the server using the /initiate-upload endpoint
const response = await fetch(`<BACKEND_URL>/initiate-upload`);
const resJson = await response.json();
const token = resJson.uploadToken;

let currentlyUploaded = 0; // Initialize a variable to keep track of the uploaded size

// Use the upload method to upload files from the browser to IPFS
const { uploadId, bucketId, protocolLink, dynamicLinks } = await upload(files, {
  token, // Pass the uploadToken for authorization
  onChunkUploaded: (uploadedSize, totalSize) => {
    currentlyUploaded += uploadedSize;
    console.log(`Uploaded ${currentlyUploaded} of ${totalSize} Bytes.`);
  },
});

2. Encrypt & Upload your Files

To encrypt and upload data to IPFS, you can use the encryptUpload function. It requires an instance of LitNodeClient that is already connected. Additionally, you need to provide the necessary parameters like authSig, accessControlConditions, chain, filePath, and litNodeClient.

const { uploadId, bucketId, protocolLink, dynamicLinks } = await spheron.encryptUpload({
  authSig, // (optional) The authSig of the user
  accessControlConditions, // (optional) The access control conditions for the signed token
  chain, // The chain name of the chain where the contract is deployed
  filePath, // (optional) The path of the file to encrypt and upload
  litNodeClient: client,
  configuration: {
    name: bucketName, // The name of the bucket on which you are uploading the data
  },
});

3. Decrypt your Uploaded Files

To decrypt the data of a previously encrypted upload, you can use the decryptUpload function. Like the encryptUpload function, it also requires an instance of LitNodeClient that is already connected with the authSig and the IpfsCid of the uploaded data.

const decryptedData: Uint8Array = spheron.decryptUpload({
  authSig, // (optional) The authSig of the user
  ipfsCid, // The IPFS cid of the previously encrypted & uploaded data
  litNodeClient, // An instance of LitNodeClient that is already connected
});

After you start the client and server, you can open http://localhost:3000 and upload data to the specified protocol.

Step 5: Handling the Response

Upon successfully uploading the file, the upload function will provide a response containing useful information. For example, the response includes uploadId, bucketId, protocolLink, dynamicLinks, and cid. These details can be utilized for future retrieval or management of the uploaded file.

Response

interface UploadRes {
  uploadId: string;
  bucketId: string;
  protocolLink: string;
  dynamicLinks: Domain[];
  cid: string;
}

Step 6: Accessing the Uploaded Data

The protocolLink property of the object returned by the upload function is a URL that can be used to access the uploaded file. The protocol link is a unique identifier for the upload, and it can be used to access the file from any IPFS node.

For example, the following code will print the protocol link of the uploaded file

console.log(upload.protocolLink);

You can find a functional demonstration of the browser-upload process by visiting this link .

Benefits of Spheron SDK

1. Efficient Data Management: Spheron SDK offers many advantages when managing large amounts of data. With this tool, developers can efficiently ingest, process, and query data in real time, making it perfect for applications that require fast data processing and analysis. Additionally, by leveraging a browser upload SDK, developers can avoid writing complex code for file uploading, such as multi-chunking and parallelization. This simplifies the development process and reduces the risk of errors.

2. Storage Scalability: With a browser upload SDK, developers do not need to worry about storage scalability and redundancy on their servers or developing a load-balancing mechanism for their server instances. Overall, Spheron SDK provides an efficient and reliable solution for managing large datasets.

3. Separate Libraries for Different Environments - Spheron provides separate libraries for different environments, including the browser and Node.js. This helps segregate the requirement of importing additional dependencies and makes it lightweight for the frontend and backend. Also, separating libraries ensures that each environment remains unburdened by unnecessary code and logic, promoting a clean and concise architecture. As a result, frontend applications can be more agile and responsive, while backend processes benefit from enhanced efficiency and resource management.

4. Functionality integration: Spheron SDK seamlessly integrates with dApps, simplifying the process of uploading files. Developers can easily incorporate the SDK into their projects, enabling users to upload files without complex implementation or additional coding.

5. Multi-chain storage capabilities: Spheron SDK is equipped with multi-chain storage capabilities, allowing developers to upload files to different protocols and chains. This means developers can easily store files across multiple decentralized storage networks without implementing separate integrations for each chain.

6. Open-Source: Spheron SDK is open-source, meaning you can watch the code. This also fosters a community-driven development model where contributors can collaborate to improve and expand the functionality of the SDK.

7. Extensive Documentation: Spheron SDK has extensive documentation that helps developers get started quickly and learn how to use the various features and functionalities.

Conclusion

Using the Spheron Browser SDK combined with IPFS provides a powerful and efficient solution for decentralized data storage. The step-by-step guide provided in this blog demonstrates how developers can easily integrate the Spheron Browser SDK into their applications to upload and manage data on IPFS. Spheron SDK offers convenient functionalities and supports multiple protocols, allowing developers to take advantage of various decentralized storage networks seamlessly. Also, the benefits of using the Spheron Browser SDK extend. Being open-source, Spheron SDK fosters community-driven development, enabling collaboration and continuous improvement.

Resources

• Spheron SDK Documentation - https://docs.spheron.network/sdk/

• IPFS Documentation - https://docs.ipfs.tech/

• GitHub Readme -

• https://github.com/spheronFdn/sdk/blob/main/examples/browser-upload/README.md

• https://github.com/spheronFdn/sdk/tree/main/packages/storage