Decentralized programming with libp2p

Distributed everything

I thought that I would write a quick post about running libp2p. If you haven't come across it yet, then you should definitely have a look at it.

The main website is here: https://libp2p.io/

Libp2p supports multiple languages, rust, golang, javascript and so on, today I will be using javascript.

Why?

I started to explore libp2p a little while ago. The main reason was personal interest. When I started playing around with it, I figured out how useful it was for a bunch of projects I'm working on (namely project tetsuo - https://tetsuo.dev).

What is it - what does it do for me?

Libp2p is a modular networking stack that you can use in your projects. It allows a developer to quickly and easily create a project that has safe, easy, decentralized networking.

That's right decentralized.

This means that multiple libp2p nodes can discover each other without needing a centralized gateway or broker (you can have this of course, but decentralized interests me more).

If you have two nodes, they will discover each other.

If you have three nodes, they will also discover each other.

Decentralized applications

If you are following along, this means that you can run a completely decentralized topology. This also means that your entire application can be decentralized.

This is important for applications where you may only want clients to connect to each other.

It is also important where you want clients to discover and automatically connect to each other.

Some of the use cases here are for things like:

• Completely decentralized peer to peer applications.
• Cluster services.
• Disconnected applications.

...the possibilities are endless.

What am I showing you today?

Today, I am going to walk through the code to be able to get a basic node to node automatic discovery happening.

Set up a project

To start with, I need to initialize a project. Since I'm using javascript, we will go ahead and start.

# create a directory for the project
mkdir test
cd test

# Initialize the project
npm init es6

Accept the defaults for your npm init. This will create a package.json file. We are good to go from here.

We are initializing the project as a module.

Get the scaffold running

To begin with, we will be creating a scaffold. All this will do at this point is create a libp2p node.

Install the pre-requisites

npm install @libp2p/tcp 
npm install libp2p

Place the code below into a file called test.js

import { tcp } from '@libp2p/tcp'
import { createLibp2p } from 'libp2p'

const createNode = async () => {
  const node = await createLibp2p({
    addresses: { listen: ['/ip4/0.0.0.0/tcp/0'] },
    transports: [ tcp() ],
  }) //end node

  return node
} //end createNode function

;(async () => {
  const [node] = await Promise.all([
    createNode(),
  ]) //end node creation as promise

  console.log('listening on addresses:')
    node.getMultiaddrs().forEach((addr) => {
      console.log(addr.toString())
  }) //end logging
})()

The code creates a new libp2p node that listens on all addresses, using TCP.

const createNode = async () => {
  const node = await createLibp2p({
    addresses: { listen: ['/ip4/0.0.0.0/tcp/0'] },
    transports: [ tcp() ],
  }) //end node

  return node
} //end createNode function

Then there is a main function that creates the node and prints out all of the addresses that the node is listening on.

;(async () => {
  const [node] = await Promise.all([
    createNode(),
  ]) //end node creation as promise

  console.log('listening on addresses:')
    node.getMultiaddrs().forEach((addr) => {
      console.log(addr.toString())
  }) //end logging
})()

You can run this using the command

node test.js
# node test.js
listening on addresses:
/ip4/127.0.0.1/tcp/33389/p2p/12D3KooWCR4TBnsBq6S7gkJuxSMFCBsQmXT4V4Z5hpEwbixsE6Va
/ip4/10.1.1.150/tcp/33389/p2p/12D3KooWCR4TBnsBq6S7gkJuxSMFCBsQmXT4V4Z5hpEwbixsE6Va
/ip4/192.168.224.129/tcp/33389/p2p/12D3KooWCR4TBnsBq6S7gkJuxSMFCBsQmXT4V4Z5hpEwbixsE6Va

Node discovery

So far we have a single libp2p node that listens. This is not that useful. Next step is to allow our node to discover other nodes on the network.

Today, we will use the multicast DNS module for this. The multicast DNS module is limited to local networks, but works well for testing and understanding.

Install the multicast DNS module.

npm install @libp2p/mdns 

Update the test.js file with the following content.

import { mdns } from '@libp2p/mdns'
import { tcp } from '@libp2p/tcp'
import { createLibp2p } from 'libp2p'

const createNode = async () => {
  const node = await createLibp2p({
    addresses: { listen: ['/ip4/0.0.0.0/tcp/0'] },
    transports: [ tcp() ],
    peerDiscovery: [ mdns() ] //default discovery period 10 seconds
  }) //end node

  return node
} //end createNode function

;(async () => {
  const [node] = await Promise.all([
    createNode(),
  ]) //end node creation as promise

  console.log('listening on addresses:')
    node.getMultiaddrs().forEach((addr) => {
      console.log(addr.toString())
  }) //end logging

  node.addEventListener('peer:discovery', (evt) => console.log('Discovered:', evt.de
tail.id.toString()))
})()

We have added some pieces to our codebase. We have added a peerDiscovery piece into the createNode async function. This essentially enables peer discovery using multicast DNS.

There are other options for peer discovery, but in this example, I am using multicast DNS.

const createNode = async () => {
  const node = await createLibp2p({
    addresses: { listen: ['/ip4/0.0.0.0/tcp/0'] },
    transports: [ tcp() ],
    peerDiscovery: [ mdns() ] //default discovery period 10 seconds
  }) //end node

This means that your node will now automatically listen on all addresses, and will perform a peer discovery search via multicast every 10 seconds. The 10 second time is fine for testing, but can be extended for real scenarios.

At the end of our main async function we place the following.

;(async () => {
  node.addEventListener('peer:discovery', (evt) => console.log('Discovered:', evt.detail.id.toString()))
})()

This piece of code adds a listener that listens specifically for peer discovery events.

Running the new code

Let's run the new codebase.

# node test.js
listening on addresses:
/ip4/127.0.0.1/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
/ip4/10.1.1.150/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
/ip4/192.168.224.129/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R

At this point, it looks exactly the same as the last time we ran it. The reason for this is that we need to run two nodes in order for us to discover anything.

Let's fire up a second instance of test.js in a separate window

node1 output

# node test.js
listening on addresses:
/ip4/127.0.0.1/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
/ip4/10.1.1.150/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
/ip4/192.168.224.129/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R

node2 output

# node test.js
listening on addresses:
/ip4/127.0.0.1/tcp/35725/p2p/12D3KooWPJSvTFo7jhaLACx1vZVKq7bHhNHKXALXWCjKqFu5LK47
/ip4/10.1.1.150/tcp/35725/p2p/12D3KooWPJSvTFo7jhaLACx1vZVKq7bHhNHKXALXWCjKqFu5LK47
/ip4/192.168.224.129/tcp/35725/p2p/12D3KooWPJSvTFo7jhaLACx1vZVKq7bHhNHKXALXWCjKqFu5LK47
Discovered: 12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R

You will note that the second node has automatically discovered the first node.

Discovered: 12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R

node1 output - check again

If you check the output of node1 again, you will note that it too has automatically discovered the other node.

# node test.js
listening on addresses:
/ip4/127.0.0.1/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
/ip4/10.1.1.150/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
/ip4/192.168.224.129/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
Discovered: 12D3KooWPJSvTFo7jhaLACx1vZVKq7bHhNHKXALXWCjKqFu5LK47

Each node is broadcasting and discovering it's peer on the network.

Full code

The full code is listed below.

import { mdns } from '@libp2p/mdns'
import { tcp } from '@libp2p/tcp'
import { createLibp2p } from 'libp2p'

const createNode = async () => {
  const node = await createLibp2p({
    addresses: { listen: ['/ip4/0.0.0.0/tcp/0'] },
    transports: [ tcp() ],
    peerDiscovery: [ mdns() ] //default discovery period 10 seconds

//    streamMuxers: [ yamux(), mplex() ],
//    connectionEncryption: [ noise() ],
  }) //end node

  return node
} //end createNode function

;(async () => {
  const [node] = await Promise.all([
    createNode(),
  ]) //end node creation as promise

  console.log('listening on addresses:')
    node.getMultiaddrs().forEach((addr) => {
      console.log(addr.toString())
  }) //end logging

  node.addEventListener('peer:discovery', (evt) => console.log('Discovered:', evt.detail.id.toString()))

// ADD YOUR APPLICATION CODE HERE

})()

What Now?

This project is a scaffold of how you can set up node discovery using libp2p. Now that you have a scaffold running, you can add your application code to the scaffold.

In my case, I am writing a cluster mechanism for application servers to automatically exchange configuration events. Every member of my application cluster will automatically receive updates to its configuration no matter which node receives the update.

In order to do this, I will be using a peer to peer message bus (also another feature of libp2p). Look out for my next post where I walk through how to get pub sub events working!