Jesal Gadhia

Posted on Apr 8, 2018 • Originally published at jes.al

Building an app with Stellar and IPFS

#node #stellar #blockchain #ipfs

While Ethereum has been the platform of choice for writing dApps, Stellar arguably has really low transaction fees and is much faster than other blockchains (including Ethereum).

So I began to wonder what it would look like to actually build a decentralized version of a forum like HackerNews or Reddit using the Stellar blockchain. Here's the big picture of how I envisioned it to work:

Let's see how we would go about implementing that.

First, we need to create an account on the Stellar testnet. What's a testnet? In the simplest terms, it's a blockchain intended for testing where you don't incur any real fees. In this case, we'll load up our test account with 10k fake lumens for testing.

Next, we will build a small JavsScript client which will allow the user to submit their post on the app.

We could directly take this post and have the user send it to our app's account by putting it in the transaction's memo field. Although it turns out Stellar's transaction only allows limited memo formats - Text (UTF-8 string of up to 28 bytes), ID (Unsigned 64-bit integer) or Hash (32-byte hash in hex format). So storing a large amount of text or JSON is out of the question.

Send it to IPFS

That's where IPFS comes in - a P2P protocol and network designed to store and share content in a distributed file system across all devices (think of it as a love child of git and BitTorrent).

We would take that data and store it in a JSON object in the IPFS.

import ipfsAPI from 'ipfs-api'

// I'm just using an IPFS gateway here for testing but in a real-world setting, we would run our own IPFS node so we can persist data
const ipfs = ipfsAPI({ host: 'ipfs.infura.io', port: 5001, protocol: 'https' });
const post = JSON.stringify({title: 'So exited!!!', content: 'This is my first post on the blockchain!', username: 'h4ck3r'})
const buffer = Buffer.from(post);

ipfs.files.add(buffer, { pin: false }, (err, ipfsHash) => {
  console.log(ipfsHash[0].path) // => QmV3C3HFE8824KWYTMq5fbZyF93GTMz5W7h3uBG1oVZCv8
});

Now we have a hash small enough to send in the memo field. Although looks like there might be another issue. IPFS represents the hash of files and objects using a Multihash multiformat with Base58 encoding. The prefix Qm corresponds to the algorithm (SHA-256) and length (32 bytes) used by IPFS.

So it looks like we are not going to be able to add this in our transaction's Text field which only allows strings of up to 28 bytes nor are we able to use the Hash field which only allows 32-byte hash.

So we'll have to write a function that converts this IPFS hash back to 32 byte hash in hex format:

import bs58 from 'bs58'

this.getBytes32FromIpfsHash = (ipfsListing) => {
  // Decode the base58 string and then slice the first two bytes
  // which represent the function code and it's length, in this case:
  // function:0x12=sha2, size:0x20=256 bits
  return bs58.decode(ipfsListing).slice(2).toString('hex')
}

Add it on the Blockchain

Now that we have the right hash to store in the memo field, we'll have to figure out how to actually send this transaction. One option is to prompt the user to make use of MetaPay which is a Chrome extension wallet for Stellar (kind of like MetaMask for Stellar Lumens). Once they have MetaPay installed, we can just setup a URL like this:

<a ref='savePost' data-meta-pay
    href="https://stellar.meta.re/transaction?to=[address]&amount=1&memo=[txMemo]"
    target="_blank" >Save Post</a>

Now if we put it all together, the submit post logic would look something like this:

import StellarSdk from 'stellar-sdk'

// Add the post data to IPFS
this.submitPost = (post) => {
  const buffer = Buffer.from(post);
  ipfs.files.add(buffer, (err, ipfsHash) => {
    this.txMemo = this.getBytes32FromIpfsHash(ipfsHash[0].path)
    this.refs['savePost'].click() // This will open the MetaPay popup
    this.confirmPayment(this.txMemo) // Listen to see if the transaction went through
  });
}

// Check to see if the transaction went through
this.confirmPayment = (ipfsHash) => {
  const server = new StellarSdk.Server('https://horizon-testnet.stellar.org');

  server.transactions().forAccount('OUR_ACCOUNT_ID').cursor('now').stream({
    onmessage: (transaction) => {
      if(transaction.memo == ipfsHash) {
        // Yes, it made it on the blockchain!
        transaction.operations().then((ops) => {
          var payment = ops._embedded.records[0];
          if(parseInt(parseFloat(payment.amount)) < 1) { 
            console.error('Payment insufficient. Post not saved!');
          } else { 
            this.pinIpfsListing(ipfsHash); 
          }
        }).catch((error) => {
          error.target.close(); // Close stream
          console.error('Payment Error: ', error);
          alert('Error confirming payment. Try again later');
        });
      }
    },
    onerror: (error) => {
      error.target.close(); // Close stream
      console.error('Streaming Error: ', error);
    }
  });
}

That will open the MetaPay popup with all the prefilled fields, we will wait and see if the user goes through with that transaction, if they do, we move to the next step.

Persist it on IPFS

// If successful, pin our post on the IPFS node
this.pinIpfsListing = (ipfsHash) => {
  ipfs.pin.add(ipfsHash)
}

Notice when we added our data to IPFS, we didn't pin it. Without pinning the post, our data will not be stored permanently on the IPFS node and will eventually be garbage collected.

So in a way that small transaction fee helps us pay for pinning the data / the cost of running an IPFS node and make sure the data is available for all users.

Read from the Blockchain & find it on IPFS

Now when other users visit the app, we will pull all the transactions posted to this app's account, grab the memo field, encode it back to Base58 and pull the data from IPFS:

import StellarSdk from 'stellar-sdk'
import ipfsAPI from 'ipfs-api'

this.getIpfsHashFromBytes32 = (bytes32Hex) => {
  // Add our default ipfs values for first 2 bytes:
  // function:0x12=sha2, size:0x20=256 bits
  const hashHex = "1220" + bytes32Hex
  const hashBytes = Buffer.from(hashHex, 'hex');
  const hashStr = bs58.encode(hashBytes)
  return hashStr
}

const server = new StellarSdk.Server('https://horizon-testnet.stellar.org');
const ipfs = ipfsAPI({ host: 'ipfs.infura.io', port: 5001, protocol: 'https' });
let posts = [];

server.transactions()
  .forAccount('OUR_ACCOUNT_ID')
  .order('desc')
  .call()
  .then((page) => {
    page.records.forEach(record => {
      if (record.memo) {
        const ipfsListing = this.getIpfsHashFromBytes32(record.memo)
        ipfs.files.get(ipfsListing, function (err, files) {
          files.forEach((file) => {
            const post = file.content.toString('utf8')
            posts.push(post) // Show this to the user
          })
        })
      }
    });
});

Decentralization

This architecture makes sure our data is decentralized but what about the app itself? If the app goes down, users could write another client that can read from that account's blockchain and pull the corresponding data from IPFS.

Although we could go one step further and actually store the client code on the blockchain as well by utilizing the manageData account operation.

Something like this could be part of the build / deploy chain for the app, so everytime a new version is released, it's also added to the blockchain:

import fs from 'fs'

this.publishClient = () {
  const code = fs.readFileSync('my_project/client.js');

  ipfs.files.add(buffer, (err, ipfsHash) => {
    const server = new StellarSdk.Server('https://horizon-testnet.stellar.org');

    server.loadAccount('OUR_ACCOUNT_ID').then((base) => {
        const tx = new StellarSdk.TransactionBuilder(base);
        const data = {name: 'v1.0.0', value: ipfsHash[0].path};
        tx.addOperation(StellarSdk.Operation.manageData(data));
        var builtTx = tx.build();
        builtTx.sign(StellarSdk.Keypair.fromSecret('OUR_ACCOUNT_SECRET'));
        return server.submitTransaction(builtTx);
    });
  });
}

Although something to keep in mind, each DataEntry increases the minimum balance needed to be held by the account. So we might want to only maintain the last version or the last couple of versions of the client codebase on the account. But that should suffice to make our demo app more or less decentralized.

Conclusion

This was an interesting thought experiment but this demo app still does not have a way to manage comments, upvotes, etc since we are somewhat limited by what the Stellar platform is capable of.

To build something more advanced we'd need to build it on a true dApp platform like Ethereum or NEO that have all the necessary tooling to make that happen.

But with the recent controversy of Facebook data and user privacy, it's definitely time to think about how to build social apps that are decentralized.

There's a lot of interesting work done in this space with projects like Datawallet, Blockstack, Akasha and others which will be interesting to follow in the coming years.

This post was originally published on my blog. If you liked this post, please share it on social media and follow me on Twitter!

Top comments (13)

Ben Halpern • Apr 8 '18

Super great. Would totally read more posts like this.

decentralized version of a forum like HackerNews or Reddit using the Stellar blockchain

dev.to is a forum like HackerNews or Reddit and thought experiments in this realm are really exciting.

Jochem Stoel • Apr 10 '18

I agree with Ben. Interesting read.

Kamal Mustafa • Apr 10 '18

Using blockchain for forum like application seem like overkill. On scuttlebutt, a forum is just an entry to the append only log with common tags.

Kamal Mustafa • Apr 12 '18

@jesalg there's patchfoo where you can use normal browser to connect to the local port patchfoo is running. It's basically a simple webapp that connect to the ssb daemon and allow you to interact with ssb network through normal browser.

Jesal Gadhia • Apr 12 '18

I'm not too familiar with scuttlebutt but a lot of the apps I saw like Patchwork seem to be standalone apps that run outside of the browser. Does the platform support web apps?