I want to welcome you to libp2p! This guide will walk you through setting up a fully functional libp2p node, so that you can take control of the spacecraft 🚀 from that point on and build your application.
Background
P2P based systems and applications have been around for several decades now. Such projects have well known challenges and difficulties that need to be addressed regarding their networking layer.
Over the years, new projects have implemented p2p networking primitives again and again. These projects usually had their networking layer tight with the application layer and were not open source. Moreover, some projects did not even reach the application layer development as a consequence of all the networking challenges that needed to be solved before getting into it.
All things considered, during these decades the available projects did not contribute to a rapid growth of the p2p ecosystem. One of the primary goals of libp2p is to fill this gap by enabling application and system developers to focus on their use cases and business logic while relying on a composable networking stack.
What is libp2p?
Libp2p is a composable and modular networking stack, which was designed to be able to become the networking layer of any type of p2p system. Its modular nature is the key to enable such a variety of projects, while also being upgradable for future network requirements.
Understanding where libp2p fits is important to understand how it is designed and how it can be used. Different p2p systems have different needs and requirements, while they might run in different environments and have different hardware capabilities.
Considerations before starting
Before you start with libp2p, it is important to understand what environments you expect your application to run on. Nowadays, you can run JavaScript code in several different environments, some of them with their own particularities.
While the libp2p core codebase aims to work in multiple environments, there are some limitations that are not possible to overcome at the time of writing. As an example, browsers do not support listening for connections via WebSockets, but only to dial known addresses, as they are designed for the traditional client-server model. As a consequence of these limitations, some environments will require to think about a network topology and additional infrastructure as a way to achieve connectivity, as well as to provide a smoother experience to the end user.
While the libp2p core codebase aims to work in multiple environments, there are some limitations that are not possible to overcome at the time of writing. As an example, browsers do not support listening for connections via WebSockets, but only to dial known addresses, as they are designed for the traditional client-server model. As a consequence of these limitations, some environments will require to think about a network topology and additional infrastructure as a way to achieve connectivity, as well as to provide a smoother experience to the end user.
Create your libp2p node with js-libp2p
It is usually recommended to start with a base configuration of libp2p and build upon it, specially for new users that are not entirely familiar with every single piece of the libp2p stack. The most important documents to have handy are the API documentation and the Configuration documentation. The former is divided into two main parts, one for creating a libp2p node and one to interact with it, while the latter is focused on configuring each type of libp2p module. The configuration of libp2p is a key part of its structure. Each project should go through what libp2p can offer and select what is important for them. Regardless of how it is configured, its top level API will always be the same.
The first step is to install libp2p in your project with your favourite JavaScript Package Manager. In this guide, we will configure the libp2p node in several stages, as this can make troubleshooting configuration issues much easier.
It is important to specify that the following content is based on libp2p@0.29 as the way libp2p is configured might change in upcoming releases.
Basic setup
With libp2p installed, let’s configure the absolute minimum needed to get your node running. The only modules libp2p requires are a Transport and a Crypto module. Moreover, it is highly recommend that a basic setup should also have a Stream Multiplexer configured. The create options can be checked in the API.md#create.
Transport
Libp2p uses Transports to establish connections between peers over the network. You can configure one Transport, or as many as you like. Supporting more Transports will improve the ability for other nodes on the network to communicate with you.
You should select Transports according to the runtime where your application will run. You can see a list of some of the available Transports. As an example, in a browser environment libp2p-websockets and libp2p-webrtc-star are typically installed.
If you need a transport that is still not available, you can easily create your own transport. The only requirement of a libp2p transport is to be compliant with the transport interface.
Crypto
In a p2p context, encryption is an essential component for the communication on the libp2p network. Every connection is required to be encrypted to ensure security for everyone.
libp2p-noise is the recommended module to use at the moment. Secio is being deprecated but other encryption modules might appear soon.
If you want to use your own connection encryption module, you just need to guarantee that it is compliant with the crypto interface.
Multiplexers
Multiplexers are not required by libp2p, but they are super important for the efficiency of the connections using several protocols that libp2p will run. It allows libp2p to run several of its internal protocols, like Identify, as well as allow your application to easily run any number of protocols over a single connection.
Running libp2p
With a Transport, Crypto and Stream Multiplexer modules specified, you can start your libp2p node. However, you also need to specify its listening addresses, so that your transports can listen for connections from other peers. Otherwise, your node will be a "dial only" peer.
This will work perfectly in a Node.js environment, but it will not be able to start in a browser context. The listen multiaddr /ip4/127.0.0.1/tcp/8000/ws aims to listen for new websocket connections on the local 8000 port. As previously stated, browsers do not support listening for connections. As a result, this type of multiaddr should not be used if your application will run in a browser environment. However, you can still use the websockets transport to dial other peers that can listen for websocket connections.
The described issue is one of the main reasons to have libp2p-webrtc-star transport. The other multiaddr /dns4/wrtc-star1.par.dwebops.pub/tcp/443/wss/p2p-webrtc-star aims to listen on a different machine that can listen for new connections. The star server on wrtc-star1.par.dwebops.pub will listen for new connections and relay them to the appropriate peer. More than listening from connection on behalf of other peers, the star server acts as a peer discovery mechanism and informs the connected peers once a new peer joins. Bear in mind that this is a public available star server and its availability is not guaranteed. You should setup your own for production.
You can check the complete API documentation o understand what you can do with your node from now on.
Customize your js-libp2p node
Now that you can have connectivity with other nodes in the network, it is time to select the modules that you need for your application, as well as to tune the node according to your needs.
There are several other modules that can be added, in order to fulfil the needs of a p2p application. These include Peer Discovery, Pubsub, Content Routing and Peer Routing, among others. Bear in mind that you should only add modules you need.
Peer Discovery
One of the most important characteristics of a typical p2p application is its ability to find other peers in the network. Libp2p provides several discovery protocols. Some of them are also dependent of the runtime of the node, like libp2p-mdns, which will not work in the browser.
You can also provide your configuration for each peer discovery module you provide. It is also important to point out that transports with built in peer discovery capabilities (e.g libp2p-webrtc-star) are automatically added as discovery modules (but can be disabled).
With Peer Discovery modules added, libp2p will handle everything and add the discovered peers to its Peer Store.
Pubsub
Some applications will potentially need a pubsub message service for real-time communication. There are two pubsub routers available, but libp2p-gossipsub is the recommended router to use thanks to its efficiency and security. Once you have a pubsub router in your node, you are able to use the Pubsub API Methods, where the most common methods are subscribe and publish.
Content and Peer Routing
Content routing offers a way to find where content lives in the network, while Peer Routing provides a mechanism to find other peers in the network by making queries using a Peer Routing algorithm. There are currently two implementations for each of these, the DHT (Distributed Hash Table) which provides both content and peer routing, and one delegate router implementation for content routing and one for peer routing. The delegate routers aim to leverage more capable nodes to issue DHT queries on behalf of the requesting peer. It is common to leverage go-ipfs nodes for this purpose.
You can have multiple routers, as well as the DHT at the same time. If that is the case, libp2p will start by trying the DHT. If it is not successful, it will try each router until it succeeds.
It is important pointing out that peers with less capabilities are not recommended to use the DHT.
Create your own protocol
So far, we went through the basics for building a regular p2p app. We may decide which modules to use according to our needs, or even create new ones. But, libp2p also allows the creation of new protocols.
Libp2p is a "modular system of protocols", where custom protocols operate in a similar fashion as a RESTful API. Each protocol has an identifier and an handler. As an example, IPFS uses the ipfs-bitswap protocol on top of libp2p for custom data exchange between peers.
After you have your custom protocol, you just need to hook it into libp2p. The handle will listen for stream messages under the protocol and call the provided handler.
What is next?
Once you have your networking building blocks ready to be used by your app, you can start focusing on it! 🎉 But before going into production, you should setup your own infrastructure. The code examples here use public infrastructure provided by Protocol Labs for experimenting, but they are not meant to be used in production and their availability is not guaranteed. According to your requirements and used modules, you might need to setup webrtc-star servers, delegate routers, or even circuit relay nodes.
There are a few new features expected for the upcoming libp2p releases that will help with some of the pain points covered here regarding connectivity and discoverability. Stay tuned and let’s distribute the web together! ❤️
At MOXY we’ve been contributing to libp2p and also leveraging it to deliver distributed applications, namely Nomios and Discussify, among a few others underway. If you are looking for help with a project that could benefit from p2p capabilities, tell us a bit more.
If you have any question, issue, or even new ideas with libp2p, you are welcome to reach out to me directly on Twitter, create a topic in the libp2p discussion forum or open Github issues in any libp2p repo.
Written by Vasco Santos
Top comments (0)