Building A P2P Network With Deno 🌐

Developing a peer-to-peer system in JavaScript, this is possible with Deno!

Recently, I discovered the SolidJS framework, and I very quickly wanted to put into practice what I had learned by developing a robust application. For this, I decided to code a web interface to control the actions of a node in a peer-to-peer network. Okay, but what's a peer-to-peer network?

A P2P system (or peer-to-peer network) is a technical data exchange architecture in which each "user" is both a client, but also a server. In this kind of structure, the user is called "node" or even "peer". There are 2 types of P2P networks:

• Centralized systems, having a main server, source of truth of the application data
• Decentralized systems, where each node guarantees the integrity and authenticity of the data

NB: It was through my interest in the concepts of "blockchain" and decentralized applications (DApps) that I decided to set up this project and to create a peer-to-peer network, from scratch.

Naturally, I immediately sought to develop this kind of system in... JavaScript! That's indeed possible, thanks to the WebSocket protocol. In this post, I propose to build, step by step, a (decentralized) P2P network based, not on ExpressJS x SocketIO, nor NestJS (to speed up the project), but directly from Deno 🦕

Part I - Server Initialization

NB: I've already mentioned what Deno is in a previous post, I invite you to take a look if you want to know more, and what differences exist with its counterpart Node.

In a P2P system, a node is both client and server. So, first step: "set up the server"; let's go!

import { serve } from 'https://deno.land/std@0.159.0/http/server.ts';

async function handler(req: Request): Promise<Response> {
  return new Response(
    JSON.stringify({
      message: 'Hello World'
    })
  );
}

serve(handler, { port: 3030 });

server.ts

Like ExpressJS, these few lines are enough to initialize an HTTP server, in order to get the "Hello World" message (in JSON format) from any route. Test it by making a call to http://localhost:3030 directly from your browser, after starting your server: deno run --allow-net server.ts

NB: Note that here I use version 1.26.0 of Deno, as well as version 0.159.0 of its modules. Deno is secure by default, so the --allow-net parameter is required to allow network access.

First API

Time to add some logic to this simple server. We will therefore create a first variable to store WebSocket clients / peers specific to the current node. As a reminder, in a decentralized model, each peer guarantees the integrity and authenticity of its data.

import { serve } from 'https://deno.land/std@0.159.0/http/server.ts';

let allPeers = [];

async function handler(req: Request): Promise<Response> {
  const url = new URL(req.url);

  if (url.pathname === '/peers' && req.method === 'GET') {
    return new Response(JSON.stringify(allPeers));
  }

  return new Response(
    JSON.stringify({
      message: 'Hello World'
    })
  );
}

serve(handler, { port: 3030 });

server.ts

Here is how to simply expose an API with Deno, without using an additional library (Alosaur, Servest, etc...). Here, we make available the /peers route to retrieve the list of peers associated with the server. This time, start your server and navigate to the URL http://localhost:3030/peers to see the result (i.e. an empty array for now...).

Part II - WebSocket Support

Let's talk about the main topic, adding support for the WebSocket standard. The server will thus be able to react in real time to the reception of data from another server on the same network.

import { serve } from 'https://deno.land/std@0.159.0/http/server.ts';

let allPeers = [];

async function handler(req: Request): Promise<Response> {
  const url = new URL(req.url);

  if (url.pathname === '/peers' && req.method === 'GET') {
    return new Response(JSON.stringify(allPeers));
  }

  const upgrade = req.headers.get('upgrade') || '';
  let response, socket: WebSocket;

  try {
    ({ response, socket } = Deno.upgradeWebSocket(req));
  } catch {
    // Request isn't trying to upgrade to WebSocket
    return new Response(
      JSON.stringify({
        message: 'Hello World'
      })
    );
  }

  socket.onopen = () => console.log('on:open');
  socket.onmessage = () => console.log('on:message');
  socket.onclose = () => console.log('on:close');
  socket.onerror = () => console.log('on:error');

  return response;
}

serve(handler, { port: 3030 });

server.ts

This "update" allows the original server to support the WebSocket protocol. For information, this Web specification allows two-way communication between a client and a server, in a single TCP request.

WebSocket Client

So far, no real change. If we start our server (deno run --allow-net server.ts) we don't see any apparent change. However, the server is now able to react to WebSocket transactions (if requested by a compatible client). Let's implement a new API that will also give more meaning to the application.

import { serve } from 'https://deno.land/std@0.159.0/http/server.ts';
import { urlWrapper } from './utils.ts';

let allPeers = [];

async function handler(req: Request): Promise<Response> {
  const url = new URL(req.url);

  // ...

  if (url.pathname === '/peer' && req.method === 'POST') {
    const body = await req.json();

    if (!body.targetUrl) {
      return new Response(JSON.stringify({ message: 'URL Value Fail' }), {
        status: 400,
      });
    }

    const [_, wsUrl] = urlWrapper(body.targetUrl);
    const wsClient = new WebSocket(wsUrl);

    wsClient.onopen = () => {
      allPeers = [...allPeers, wsClient];
    };

    wsClient.onclose = () => {
      allPeers = allPeers.filter((client) => {
        const reUrl = new RegExp(wsUrl);
        return !reUrl.test(client.url);
      });
    };

    return new Response(JSON.stringify({ message: `New Peer: '${wsUrl}'` }), {
      status: 201,
    });
  }

  // ...

  return response;
}

serve(handler, { port: 3030 });

server.ts (code snippet)

Here is the second API to create a new WebSocket client. Explicitly, the user will need to provide the client node's target URL via a REST protocol POST call. When the client is opened (onopen), it is added to the peer list of the current server; on the other hand, when the client is closed (onclose), this same client is removed from the list. Finally, the API returns a simple message with a status 201 (CREATED), or an HTTP 400 code (BAD REQUEST) if the URL value isn't present in the body of the call. Since this is a POST request, it is necessary to use a REST client: cURL, Postman, etc...; to query the API.

/**
 * @param {string} url
 * @returns {string[]} HTTP URL + WS URL
 */
export const urlWrapper = (url: string) => {
  // That's an HTTP URL
  if (/^(http:\/\/)/.test(url)) {
    const wsUrl = url.replace('http://', 'ws://');
    return [url, wsUrl];
  }

  // That's an WS URL
  if (/^(ws:\/\/)/.test(url)) {
    const httpUrl = url.replace('ws://', 'http://');
    return [httpUrl, url];
  }

  // There's no protocol...
  return [`http://${url}`, `ws://${url}`];
};

utils.ts

NB: I use an intermediate function to format the URL. Indeed, the creation of a new WebSocket client requires the presence of its protocol (ws://) rather than the character string specific to an HTTP URL (http://). My method (although perfectible) can retrieve one or the other, depending on the value entered by the user when querying the API.

To test this second API, I invite you to follow the following process:

1. In a terminal, start a server (on the default port 3030)
2. Change the port (4040), then start a new server in a 2nd terminal
3. Finally, from a REST client, query the API /peer as POST, from the 1st server (http://localhost:3030/peer), with the body { "targetUrl": "127.0. 0.1:4040" }

curl -H "Content-Type: application/json" -X POST -d '{"url": "127.0.0.1:4040"}' http://localhost:3030/peer

POST request via cURL

If everything goes well, you should see the 2nd terminal (the one whose server is started on port 4040) display an "on:open" message. This information can be explained by the fact that the 1st server opened a client to the second. Similarly, if you ask the 1st server to find out the list of its nodes, it should have completed the list with the following peer: ws://127.0.0.1:4040

curl -H "Content-Type: application/json" -X GET http://localhost:3030/peers

GET request via cURL

Okay, but... Where are we with all this!? At the end of this chapter, you should have two "nodes" of your P2P network (if you have followed the tutorial carefully, and you haven't started a 3rd instance):

• A "fake" node started on port 3030 which is only server
• A "real" node started on port 4040 which is both client (from the first node) and server

In summary, the 1st server is able to send and receive data to 4040. On the other hand, the 2nd server receives the data but cannot send any in turn to 3030. Conclusion, we aren't yet in a peer-to-peer system...

Part III - The "Handshake" 🤝

Now, what remains to be done (compared to the previous part) is a "Handshake" 🤝 / or pairing method. This process allows a node to open a client to a server, so that the latter connects to it in turn. Let's put this in place!

Pong !

A.K.A "The Receiver"

import { serve } from 'https://deno.land/std@0.159.0/http/server.ts';
import { urlWrapper } from './utils.ts';

let allPeers = [];

async function handler(req: Request): Promise<Response> {
  const url = new URL(req.url);

  // ...

  socket.onopen = () => console.log('on:open');

  socket.onmessage = ({ data }) => {
    const parsedData = JSON.parse(data);

    switch (parsedData.type) {
      case 'HANDSHAKE':
        const [_, wsUrl] = urlWrapper(parsedData.payload);
        const wsClient = new WebSocket(wsUrl);

        wsClient.onopen = () => {
          allPeers = [...allPeers, wsClient];
        };

        wsClient.onclose = () => {
          allPeers = allPeers.filter((client) => {
            const reUrl = new RegExp(wsUrl);
            return !reUrl.test(client.url);
          });
        };
        break;

      default:
        throw new Error();
    }
  };

  socket.onclose = () => console.log('on:close');
  socket.onerror = () => console.log('on:error');

  return response;
}

serve(handler, { port: 3030 });

server.ts (code snippet)

In this 1st part of the code, the server will react to the reception of a message with the type "HANDSHAKE" 🤝, then it will open a client to the WebSocket URL of the target, previously passed as a value (payload) of this same message.

NB: With TypeScript, I advise you to use enums rather than constants for your types, it's smarter 😉

Ping !

A.K.A "The Emitter"

import { serve } from 'https://deno.land/std@0.159.0/http/server.ts';
import { urlWrapper } from './utils.ts';

let allPeers = [];

async function handler(req: Request): Promise<Response> {
  const url = new URL(req.url);

  // ...

  if (url.pathname === '/peer' && req.method === 'POST') {
    const body = await req.json();

    if (!body.targetUrl) {
      return new Response(JSON.stringify({ message: 'Target URL Value Fail' }), {
        status: 400,
      });
    }

    const [_, wsUrl] = urlWrapper(body.targetUrl);
    const wsClient = new WebSocket(wsUrl);

    wsClient.onopen = () => {
      allPeers = [...allPeers, wsClient];

      wsClient.send(
        JSON.stringify({ type: 'HANDSHAKE', payload: '127.0.0.1:3030' })
      );
    };

    wsClient.onclose = () => {
      allPeers = allPeers.filter((client) => {
        const reUrl = new RegExp(wsUrl);
        return !reUrl.test(client.url);
      });
    };

    return new Response(JSON.stringify({ message: `New Peer: '${wsUrl}'` }), {
      status: 201,
    });
  }

  // ...

  return response;
}

serve(handler, { port: 3030 });

server.ts (code snippet)

Then, we will enrich the service responsible for creating a peer / a node, so that it sends a message to the target server, when opening a new WebSocket client. Thus, each of the two servers (according to this tutorial) will be able to communicate with each other. Test it according to the process of the previous part. Again, if all goes well, each terminal should display the message "on:open" which means that each server has opened a client to the other. That's it! Your peer-to-peer system is functional 👍

Part IV - Data Broadcast

Since we have an operational peer-to-peer network, it's time to move on to sending data between nodes, especially by communicating globally. To do this, we will have to implement a new type of data, enrich this new data structure and access it via APIs, and of course, distribute them to our network.

import { serve } from 'https://deno.land/std@0.159.0/http/server.ts';
import { urlWrapper } from './utils.ts';

let allPeers = [];
let allRecords = [];

async function handler(req: Request): Promise<Response> {
  const url = new URL(req.url);

  // ...

  if (url.pathname === '/records' && req.method === 'GET') {
    return new Response(
      JSON.stringify(allRecords.map((record) => record.value))
    );
  }

  if (url.pathname === '/record' && req.method === 'POST') {
    const body = await req.json();

    if (!body.record) {
      return new Response(JSON.stringify({ message: 'Record Value Fail' }), {
        status: 400
      });
    }

    allRecords = [
      ...allRecords,
      {
        key: Math.random().toString(36).slice(2),
        value: body.record
      }
    ];

    allPeers.forEach((client) => {
      client.send(
        JSON.stringify({
          type: 'NEW_RECORD',
          payload: allRecords[allRecords.length - 1]
        })
      );
    });

    return new Response(
      JSON.stringify({ message: `New Record: '${body.record}'` }),
      { status: 201 }
    );
  }

  // ...

  return response;
}

serve(handler, { port: 3030 });

server.ts (code snippet)

First, we create two new APIs (respectively in GET and POST) for retrieving a list of records (allRecords), as well as adding a new element (record). Moreover, during the insertion of this record, a WebSocket message is sent to all peers of the current server. The information is therefore distributed in a global way.

import { serve } from 'https://deno.land/std@0.159.0/http/server.ts';
import { urlWrapper } from './utils.ts';

let allPeers = [];
let allRecords = [];

async function handler(req: Request): Promise<Response> {
  const url = new URL(req.url);

  // ...

  socket.onopen = () => console.log('on:open');

  socket.onmessage = ({ data }) => {
    const parsedData = JSON.parse(data);

    switch (parsedData.type) {
      case 'HANDSHAKE':
        const [_, wsUrl] = urlWrapper(parsedData.payload);
        const wsClient = new WebSocket(wsUrl);

        wsClient.onopen = () => {
          allPeers = [...allPeers, wsClient];
        };

        wsClient.onclose = () => {
          allPeers = allPeers.filter((client) => {
            const reUrl = new RegExp(wsUrl);
            return !reUrl.test(client.url);
          });
        };
        break;

      case 'NEW_RECORD':
        const foundedRecord = allRecords.find(
          (record) => record.key === parsedData.payload.key
        );

        if (!foundedRecord) {
          allRecords = [...allRecords, parsedData.payload];
        }
        break;

      default:
        throw new Error();
    }
  };

  socket.onclose = () => console.log('on:close');
  socket.onerror = () => console.log('on:error');

  return response;
}

serve(handler, { port: 3030 });

server.ts (code snippet)

On the other hand, the server receives the message of type "NEW_RECORD", then we insert the value of the message (payload) as a new element of the record list, if there isn't already listed. Note that the record is identified by a unique and random key (Thanks to the object Math 😉)

So I invite you to test this operation, by starting an additional server (in a new terminal):

1. Start a 1st server (on the default port 3030)
2. Start a 2nd server by changing the port (4040)
3. Start a 3rd server by changing the port (5050)
4. Add the 4040 and 5050 nodes to the 3030 server peer list
5. Save one or more data from the /record API of the first server
6. Check record contents from node 4040 or 5050

curl -H "Content-Type: application/json" -X POST -d '{"targetUrl": "127.0.0.1:4040"}' http://localhost:3030/peer
curl -H "Content-Type: application/json" -X POST -d '{"targetUrl": "127.0.0.1:5050"}' http://localhost:3030/peer
curl -H "Content-Type: application/json" -X POST -d '{"record": "hello"}' http://localhost:3030/record
curl -H "Content-Type: application/json" -X POST -d '{"record": "world"}' http://localhost:3030/record
curl -H "Content-Type: application/json" -X GET http://localhost:4040/records

List of cURL requests

Part V - Command-Line Interface

As you will have noticed several times, the test process is not really optimal... Changing the port by hand is relatively heavy. Don't panic, this last (bonus) part, will allow you to be more flexible in creating nodes for your peer-to-peer system.

import { parse } from 'https://deno.land/std@0.159.0/flags/mod.ts';
import { serve } from 'https://deno.land/std@0.159.0/http/server.ts';
import { urlWrapper } from './utils.ts';

const parsedArgs = parse(Deno.args);

const HOST = parsedArgs.host || '127.0.0.1';
const PORT = +parsedArgs.port || 3030;

let allPeers = [];
let allRecords = [];

async function handler(req: Request): Promise<Response> {
  // ...

  wsClient.send(
    JSON.stringify({ type: 'HANDSHAKE', payload: `${HOST}:${PORT}` })
  );

  // ...

  return response;
}

serve(handler, { hostname: HOST, port: PORT });

server.ts (code snippet)

These few additional lines of code should allow you to manipulate the host and the port more easily: deno run --allow-net server.ts --port 5050. Let's go further in this same logic by allowing the addition of peers (already started) when the server is launched.

import { parse } from 'https://deno.land/std@0.159.0/flags/mod.ts';
import { serve } from 'https://deno.land/std@0.159.0/http/server.ts';
import { urlWrapper } from './utils.ts';

const parsedArgs = parse(Deno.args);

const HOST = parsedArgs.host || '127.0.0.1';
const PORT = +parsedArgs.port || 3030;

let allPeers = [];
let allRecords = [];

async function handler(req: Request): Promise<Response> {}

serve(handler, {
  hostname: HOST,
  port: PORT,
  onListen: ({ hostname, port }) => {
    console.log(`Server started on http://${hostname}:${port}`);
  }
});

if (parsedArgs.peers) {
  const parsedPeers: string[] = parsedArgs.peers.split(',');

  parsedPeers.forEach((peer) => {
    const [_, wsUrl] = urlWrapper(peer);
    const wsClient = new WebSocket(wsUrl);

    wsClient.onopen = () => {
      allPeers = [...allPeers, wsClient];

      wsClient.send(
        JSON.stringify({ type: 'HANDSHAKE', payload: `${HOST}:${PORT}` })
      );
    };

    wsClient.onclose = () => {
      allPeers = allPeers.filter((client) => {
        const reUrl = new RegExp(wsUrl);
        return !reUrl.test(client.url);
      });
    };
  });
}

server.ts (code snippet)

Here we are! Thanks to this last parameter you are ready to launch your P2P network 👍 Now just make it grow, little by little, by connecting the Deno 🦕 to each other. You are free to add complexity, security and why not "blockchain" concepts. Enjoy!