DEV Community: Anjan Roy

Service Discovery within LAN

Anjan Roy — Mon, 29 Apr 2019 09:57:46 +0000

idea

Let's assume a situation -- You've two devices within same Local Area Network and you want to send a file from deviceA to deviceB.

    deviceA -> deviceB

What do you do then ?

Write a server program for deviceA and one client program for deviceB 🤔.

Well this works. But next time when you run these programs, deviceA and deviceB may have different IP addresses or another deviceC may be interested in participating in file transfer, which might eventually lead you to manually updating PEER's IP address 😞.

How about we create an automated device discovery service ?

Sounds great 👏

So let's get to work 🏃

model

We're going to use DatagramSocket, for handling Service Advertising and Service Discovery with in LAN. Implementation will be done using Dart Language and snippets for JavaScript will be provided.

init

First we're going to write Service Advertiser i.e. a UDP server program.

Service Advertiser

Classes required for implementation of proposed functionality.

import 'dart:io' show RawDatagramSocket, RawSocketEvent, InternetAddress, Datagram;
import 'dart:convert' show utf8;

Binding a RawDatagramSocket on InternetAddress.anyIPv4 and port 8000, so that it can listen on all interfaces on port 8000.

RawDatagramSocket.bind(InternetAddress.anyIPv4, 8000).then((datagramSocket) {
    // more code coming ...
  });

You can see RawDatagramSocket.bind( ... ) returns a Future. We'll enable READ event listener on this socket, so that whenever any other device tries to write something on this socket, a listener will be notified.

datagramSocket.readEventsEnabled = true; // put this line within aforementioned `then()`

Now we'll listen for READ events on this
DatagramSocket. Whenever a new READ event occurs it's fetched using DatagramSocket.receive() and other end get's notified that it has reached a valid Service Advertiser.

datagramSocket.listen((RawSocketEvent event) {
      if (event == RawSocketEvent.read) {
          // checking whether it's a read event or not
        Datagram dg = datagramSocket.receive();
        if (dg != null) {
        // notifying other end that it has reached a service advertiser
          datagramSocket.send(dg.data, dg.address, dg.port);
          print('${dg.address}:${dg.port} -- ${utf8.decode(dg.data)}');
        }
      }
    });

PEER's IP address, port number and received data are extracted from Datagram object, received on occurance of READ event.

Received data is decoded back to String from List, using following function from utf8 module.

utf8.decode(dg.data);

Well Service Advertiser is completed 😉.

Service Discoverer

You may have already guessed, this will be simply a UDP client.

Classes required for implementation of service discovery functionality.

import 'dart:io' show RawDatagramSocket, RawSocketEvent, InternetAddress, Datagram;
import 'dart:convert' show utf8;

Again we need to bind a RawDatagramSocket on InternetAddress.anyIPv4 and port 0, which will return Future.

Why port 0 ? 🤔

Well specifying port number 0, lets us use OS chosen port number. In client side we're not that much interested in listening on a predefined port number 😊.

RawDatagramSocket.bind(InternetAddress.anyIPv4, 0).then((datagramSocket) {
    // more code coming ...
  });

During Service Advertising, we were just listening for READ events on RawDatagramSocket, but now we're also going to enable broadcast, so that this socket can send a broadcast message to 255.255.255.255, which will reach all devices within LAN.

// put these two lines with in previous `then()`
datagramSocket.broadcastEnabled = true;
datagramSocket.readEventsEnabled = true;

Time to send a message on broadcastAddress and port 8000.

datagramSocket.send("io.github.itzmeanjan.transferZ".codeUnits,
        InternetAddress("255.255.255.255"), 8000);

Aforementioned message reaches all devices present in LAN listening for READ event on port 8000.

Why port 8000 ? 🤔

Well we're using port 8000 for Service Advertising 😉.

You may be wondering what's "io.github.itzmeanjan.transferZ".codeUnits, this returns List from String, to be sent as broadcastMessage.

Now let's put logic required for listening READ events on RawDatagramSocket. Following snippet is pretty close to the one which we used in Service Advertiser, except the fact that we don't send anything back to Service Advertiser.

datagramSocket.listen((RawSocketEvent event) {
      if (event == RawSocketEvent.read) {
        Datagram dg = datagramSocket.receive();
        if (dg != null) {
          print('${dg.address}:${dg.port} -- ${utf8.decode(dg.data)}');
          //datagramSocket.close();
          // You may consider uncommenting previous line, if you want to explore only one Service Advertiser at a time.
        }
      }
    });

And our Service Discoverer is complete 😃.

run

Time to run it 🏃

Service Advertiser

Run it using

$ dart service_advertiser.dart

Service Discoverer

Now run the servise discoverer using

$ dart service_discoverer.dart

Here's what happened ---

I start Service Advertiser from one terminal, another Service Advertiser from a Flutter mobile app and a Service Discoverer from another terminal on same computer. All these devices are with in same LAN.

First our Service Discoverer sends a broadcastMessage to 255.255.255.255 on port 8000, and starts listening for READ event(s) on RawDatagramSocket. Then Service Advertiser(s) receive READ event notification and responds to it. Response comes to Service Discoverer, it reads that and prints out on screen.

end

And we've successfully implemented nearby service advertising & discovery with Datagram Socket, implemented using Dart Language.

Interested in having this implementation in JavaScript ?

Check here for service_advertiser and service_discoverer.

If you are interested in learning how to use this with a flutter app, you might consider checking transferZ, an android app built with ❤️ using Flutter for sending files to devices with in LAN.

See you in next article. In the mean time you may consider following me on GitHub and Twitter.

Wrap up time ✌️.

Working with WebSocket

Anjan Roy — Fri, 26 Apr 2019 20:54:37 +0000

init

WebSocket -- used in real-time data transfer scenarios, such as multiplayer games/ chatbots/ sensor network in IoT etc.

define

Well, WebSocket is simply a general HTTP connection with some modifications, where a normal HTTP connection gets upgraded to WebSocket connection, when two communicating PEERs agree to use a single socket for all their communication, followed by a handshake. It could be thought of as a bidirectional communication channel in between two PEERs. When communication ends, that socket is destroyed.

idea

May be you've already worked with WebSocket using NodeJS or Java or PHP or Python or something else, but today we're going to use Dart Language for implementing both WebSocket Server & Client in a much easier way.

Alright so let's get into coding ...

implement

First we're going to write a WebSocket Server

WebSocket Server

As I've already said, WebSocket is a modified version of a HTTP connection, we're going to create a HttpServer and then we'll upgrade that connection to WebSocket.

import 'dart:io' show HttpServer, HttpRequest;
main() {
  HttpServer.bind('localhost', 8000).then((HttpServer server) {
      // httpserver listens on http://localhost:8000
    print('[+]HttpServer listening at -- http://localhost:8000');
    server.listen((HttpRequest request) {

       // more code goes here

    }, onError: (err) => print('[!]Error -- ${err.toString()}'));
  }, onError: (err) => print('[!]Error -- ${err.toString()}'));
}

Now we've HttpRequest object, which is going to be upgraded to WebSocket.

import 'dart:io' show WebSocket, WebSocketTransformer;
import 'dart:convert' show json;
import 'dart:async' show Timer;
// will put this snippet with in main()
WebSocketTransformer.upgrade(request).then((WebSocket ws) {
    // upgradation happens here and now we've a WebSocket object
        ws.listen( // listening for data
          (data) {
            print(
                '\t\t${request?.connectionInfo?.remoteAddress} -- ${Map<String, String>.from(json.decode(data))}'); // client will send JSON data
            Timer(Duration(seconds: 1), () {
              if (ws.readyState == WebSocket.open)
                ws.add(json.encode({
                  'data': 'from server at ${DateTime.now().toString()}', // in response JSON data is sent back to client
                })); // Timer helps to schedule code execution
            });
          },
          onDone: () => print('[+]Done :)'),
          onError: (err) => print('[!]Error -- ${err.toString()}'),
          cancelOnError: true,
        );
      }, onError: (err) => print('[!]Error -- ${err.toString()}'));

See it's this much easy to upgrade a HTTP connection to WebSocket connection, with Dart ❤️.

Here's our whole WebSocket Server code.

import 'dart:io' show HttpServer, HttpRequest, WebSocket, WebSocketTransformer;
import 'dart:convert' show json;
import 'dart:async' show Timer;

main() {
  HttpServer.bind('localhost', 8000).then((HttpServer server) {
    print('[+]WebSocket listening at -- ws://localhost:8000/');
    server.listen((HttpRequest request) {
      WebSocketTransformer.upgrade(request).then((WebSocket ws) {
        ws.listen(
          (data) {
            print(
                '\t\t${request?.connectionInfo?.remoteAddress} -- ${Map<String, String>.from(json.decode(data))}');
            Timer(Duration(seconds: 1), () {
              if (ws.readyState == WebSocket.open)
              // checking connection state helps to avoid unprecedented errors
                ws.add(json.encode({
                  'data': 'from server at ${DateTime.now().toString()}',
                }));
            });
          },
          onDone: () => print('[+]Done :)'),
          onError: (err) => print('[!]Error -- ${err.toString()}'),
          cancelOnError: true,
        );
      }, onError: (err) => print('[!]Error -- ${err.toString()}'));
    }, onError: (err) => print('[!]Error -- ${err.toString()}'));
  }, onError: (err) => print('[!]Error -- ${err.toString()}'));
}

Let's go for WebSocket Client.

WebSocket Client

Now Dart has got a class WebSocket in dart:io, which takes care of implementation of WebSocket Client, does all the heavy liftings for us ⚒️.

You can assume, implementation of WebSocket Client will be much easier.

import 'dart:io' show WebSocket;
import 'dart:convert' show json;
import 'dart:async' show Timer;

main() {
  WebSocket.connect('ws://localhost:8000').then((WebSocket ws) {
      // our websocket server runs on ws://localhost:8000
    if (ws?.readyState == WebSocket.open) {
        // as soon as websocket is connected and ready for use, we can start talking to other end
      ws.add(json.encode({
        'data': 'from client at ${DateTime.now().toString()}',
      })); // this is the JSON data format to be transmitted
      ws.listen( // gives a StreamSubscription
        (data) {
          print('\t\t -- ${Map<String, String>.from(json.decode(data))}'); // listen for incoming data and show when it arrives
          Timer(Duration(seconds: 1), () {
            if (ws.readyState == WebSocket.open) // checking whether connection is open or not, is required before writing anything on socket
              ws.add(json.encode({
                'data': 'from client at ${DateTime.now().toString()}',
              }));
          });
        },
        onDone: () => print('[+]Done :)'),
        onError: (err) => print('[!]Error -- ${err.toString()}'),
        cancelOnError: true,
      );
    } else
      print('[!]Connection Denied');
      // in case, if serer is not running now
  }, onError: (err) => print('[!]Error -- ${err.toString()}'));
}

Wasn't it easier than WebSocket Server implementation ?

And yes, with 930 bytes of client-side Dart code and 1175 bytes of server-side Dart code, we've completed WebSocket implementation.

run

Make sure you've installed Dart SDK from here and set PATH variable properly.

Time to run these beasts 😆

$ dart websocket_server.dart

And in another terminal

$ dart websocket_client.dart

And 💥

end

In this article, you've learnt how to use Dart for building WebSocket Peers, for tranferring real-time data.

Got any problems ?

Yes

Find me on Twitter

Wrap up time. See you in next article.

In the mean time, you may check out my GitHub profile for my other projects.

Making of transferZ (Part: 1/n)

Anjan Roy — Mon, 22 Apr 2019 10:41:01 +0000

Here we begin ...

idea

Building a flutter application for transferring files wirelessly, in between your devices, mostly written with Dart Language, can be really great.

info

transferZ is an opensource project, find whole source code here. Feel free to contribute by submitting a PR.

Now, you can download and test transferZ on your device.

Today we gonna implement Client-Server portion of this Flutter application.

And at end of this series, you'll have a working file transfer app.

Wanna do so ?

Yes

Alright follow me 😉

model

A device can act in either of these two modes at a time.

Receive
Send

In first case, device will behave like a Client and in case of later one, it'll simply be a Server.

When in Receive mode, first Client fetches a list of files, which are ready to be shared by Server.

Then it iterates through that list of files and fetches them one by one, while Server keeps listening for incoming requests and serves them, if it's from an allowed PEER.

Yes, it's this much simple 😄.

init

Let's first write some code to make our device act like a Client.

Receive Mode

Following import statements will fetch required Dart classes for us.

import 'dart:io';
import 'dart:convert' show json, utf8;
import 'dart:async';

A class, named Client will incorporate all Client functionalities, to be shown in Receive Mode.

class Client {
  String _peerIP; // this the server IP, where we'll connect
  int _peerPort; // port on which server listens

  HttpClient _httpClient;

  Client(this._peerIP, this._peerPort) {
    _httpClient = HttpClient(); // gets initialized in constructor block
  }
}

Now add a method, named connect in our Client class.

// method GETs a certain path, and returns a HttpClientRequest object, which is to be closed later, to send that request to server.
Future<HttpClientRequest> connect(String path) async => await _httpClient
      .get(this._peerIP, this._peerPort, path)
      .catchError((e) => null);

As we modeled our Client, to be first fetching a list of files, which are ready to be shared by Server, we'll require a method, named fetchFileNames, in Client class.

Future<List<String>> fetchFileNames(HttpClientRequest req) async {
    // a map, stores JSON response from server
    var fileNames = <String, List<String>>{};
    // Completer class helps us to return a Future from this method and when Completer completes, result is send back.
    var completer = Completer<List<String>>();
    await req.close().then((HttpClientResponse resp) {
      if (resp.statusCode == 200)
        // in case of success, listen for data passed through Stream
        resp.listen(
          (data) => fileNames = serializeJSON(utf8.decode(data)), // decode response and then serialize it into Map<String, List<String>> 
          onDone: () => completer.complete(fileNames['files']), // extract list of file names and send that back
          onError: (e) => completer.complete(<String>[]), // send a blank list
          cancelOnError: true,
        );
      else
        // if HttpStatus other than 200, simply return a blank list
        resp.listen(
          (data) {},
          onDone: () => completer.complete(<String>[]),
          onError: (e) => completer.complete(<String>[]),
          cancelOnError: true,
        );
    });
    return completer.future;
  }

Closing HttpClientRequest object, which was received from previous connect method, gives us a HttpClientResponse object, which is used for reading response from Server.

Now we need to define another method serializeJSON, which will help us to convert JSON response into Map < String, List< String > >.

Map<String, List<String>> serializeJSON(String data) =>
    return Map<String, dynamic>.from(json.decode(data))
        .map((key, val) => MapEntry(key, List<String>.from(val)));

What do we have now ?
A list of Strings, which are file names to be fetched from Server.

Let's fetch those files from Server. Add a new method fetchFile in Client class.

Future<bool> fetchFile(HttpClientRequest req, String targetPath) async {
    var completer = Completer<bool>(); // returns a future of boolean type, to denote success or failure
    await req.close().then((HttpClientResponse resp) {
      if (resp.statusCode == 200)
        File(targetPath).openWrite(mode: FileMode.write).addStream(resp).then(
              (val) => completer.complete(true),
              onError: (e) => completer.complete(false),
            ); // file opened in write mode, and added response Stream into file.
      else
        resp.listen(
          (data) {},
          onDone: () => completer.complete(false),
          onError: (e) => completer.complete(false),
          cancelOnError: true,
        ); // in case of error, simply return false to denote failure
    });
    return completer.future;
  }

If file is properly fetched, method returns true, else false is sent back

And a last method, named disconnect for closing HttpClient connection.

disconnect() =>
      // never force close a http connection, by passing force value attribute true.
      _httpClient.close();

And we've completed declaring client 😎.

Send Mode

Back to Server. Let's first define a class, named Server.

import 'dart:io';
import 'dart:convert' show json;

class Server {
  String _host; // IP where on which server listens
  int _port; // port on which server listens for incoming connections
  List<String> _filteredPeers; // list of IP, which are permitted to request files
  List<String> _files; // files ready to be shared
  ServerStatusCallBack _serverStatusCallBack; // Server status callback, lets user know, what's happening
  bool isStopped = true; // server state denoter

  Server(this._host, this._port, this._filteredPeers, this._files,
      this._serverStatusCallBack); // constructor defined

For controlling Server, we need to declare another variable with in this class.

    HttpServer _httpServer;

Let's start listening for incoming connections.

initServer() async {
    await HttpServer.bind(this._host, this._port).then((HttpServer server) {
    // binds server on specified IP and port
      _httpServer = server; // initializes _httpServer, which will be useful for controlling server functionalities
      isStopped = false; // server state changed
      _serverStatusCallBack.generalUpdate('Server started'); // lets user know about it, using callback function
    }, onError: (e) {
      isStopped = true;
      _serverStatusCallBack.generalUpdate('Failed to start Server');
    });
    // listens for incoming request in asynchronous fashion
    await for (HttpRequest request in _httpServer) {
      handleRequest(request); // handles incoming request
    }
  }

For handling incoming connection, add a method, named handleRequest in our Server class.

handleRequest(HttpRequest request) {
    // first checks whether device allowed to request file or not
    if (isPeerAllowed(request.connectionInfo.remoteAddress.address)) {
      if (request.method == 'GET')
        handleGETRequest(request); // only GET is permitted
      else
        request.response
          ..statusCode = HttpStatus.methodNotAllowed
          ..headers.contentType = ContentType.json
          ..write(json.encode(
              <String, int>{'status': 'GET method only'}))
          ..close().then(
              (val) => _serverStatusCallBack.updateServerStatus({
                    request.connectionInfo.remoteAddress.host:
                        'GET method only'}), onError: (e) {
            _serverStatusCallBack.updateServerStatus({
              request.connectionInfo.remoteAddress.host:
                  'Transfer Error'
            });
          }); // otherwise let client know about it by sending a JSON response, where HTTP statusCode is set as HttpStatus.methodNotAllowed
    } else
      request.response
        ..statusCode = HttpStatus.forbidden
        ..headers.contentType = ContentType.json
        ..write(
            json.encode(<String, int>{'status': 'Access denied'}))
        ..close().then(
            (val) =>
                _serverStatusCallBack.generalUpdate('Access Denied'),
            onError: (e) {
          _serverStatusCallBack.generalUpdate('Transfer Error');
        }); // if client is not permitted to access
  }

Checking if Peer can be granted access, is done in isPeerAllowed.

bool isPeerAllowed(String remoteAddress) =>
      this._filteredPeers.contains(remoteAddress); // _filteredPeers was supplied during instantiating this class

Time to handle filtered GET requests, with handleGETRequest.

handleGETRequest(HttpRequest getRequest) {
    if (getRequest.uri.path == '/') {
    // client hits at `http://hostAddress:port/`, for fetching list of files to be shared 
      getRequest.response
        ..statusCode = HttpStatus.ok // statusCode 200
        ..headers.contentType = ContentType.json // JSON response sent
        ..write(json.encode(<String, List<String>>{"files": this._files})) // response to be processed by client for converting JSON string back to Map<String, List<String>>
        ..close().then(
            (val) => _serverStatusCallBack.updateServerStatus({
                  getRequest.connectionInfo.remoteAddress.host:
                      'Accessible file list shared'
                }), onError: (e) {
          _serverStatusCallBack.updateServerStatus({
            getRequest.connectionInfo.remoteAddress.host:
                'Transfer Error'
          }); // in case of error
        });
    } else {
      if (this._files.contains(getRequest.uri.path)) {
    // if client hits at `http://hostIP:port/filePath`, first it's checked whether this file is supposed to be shared or not
        String remote = getRequest.connectionInfo.remoteAddress.host;
        getRequest.response.statusCode = HttpStatus.ok;
        _serverStatusCallBack.updateServerStatus({
          getRequest.connectionInfo.remoteAddress.host:
              'File fetch in Progress'
        });
        // then file is opened and added into response Stream
        getRequest.response
            .addStream(File(getRequest.uri.path).openRead())
            .then(
          (val) {
            getRequest.response.close(); // close connection
            _serverStatusCallBack
                .updateServerStatus({remote: 'File fetched'});
          },
          onError: (e) {
            _serverStatusCallBack
                .updateServerStatus({remote: 'Transfer Error'}); // in case of error
          },
        );
      }
    }
  }

And last but not least, a way to stop Server.

stopServer() {
    isStopped = true;
    _httpServer?.close(force: true);
    _serverStatusCallBack.generalUpdate('Server stopped');
  }

And this is all about Server 😎.

Now define an abstract class ServerStatusCallBack, to facilitate callback functionality. This abstract class needs to be implemented, from where we plan to start, control and stop Server.

abstract class ServerStatusCallBack {
  updateServerStatus(Map<String, String> msg); // a certain peer specific message

  generalUpdate(String msg); // general message about current server state
}

run

Time to run our client.

Receive Mode

Currently I'm going to run client simply from a command line app, which is to be updated in upcoming articles, when we reach UI context in flutter.

main() {
  var client = Client('x.x.x.x', 8000); // x.x.x.x -> server IP and 8000 is port number
// feel free to change port number, try keeping it >1024
  client.connect('/').then((HttpClientRequest req) {
    if (req != null) // checks whether server is down or not
      client.fetchFileNames(req).then((List<String> targetFiles) { // first fetch list of files by sending a request in `/` path
        targetFiles.forEach((path) { // fetch files, one by one
          client.connect(path).then((HttpClientRequest req) { // request for a file, with that file's name as request path
            if (req != null) { // if server finds request eligible
              print('fetching ${getTargetFilePath(path)}');
              client
                  .fetchFile(req, getTargetFilePath(path)) // fetch file
                  .then((bool isSuccess) { // after it's done, check for transfer status
                print(isSuccess
                    ? 'Successfully downloaded file'
                    : 'Failed to download file');
                if (targetFiles.last == path) {
                  print('complete');
                  client.disconnect(); // disconnect client, when we find that all files has been successfully fetched
                }
              });
            } else
              print('Connection Failed');
          });
        });
        if (targetFiles.isEmpty) {
          print('incomplete');
          client.disconnect(); // there might be a situation when client not permitted to access files, then we get an empty list of files, it's handled here
        }
      });
    else
      print('Connection Failed'); // couldn't connect to server, may be down
  });
}

// returns path where client will store fetched files
String getTargetFilePath(String path) =>
    '/path-to-target-directory-to-store-fetched-file/${path.split('/').last}';

Send Mode

In case of Server too, for time being, we'll keep it command line executable.

Let's first define a class ServerDriver, which implements ServerStatusCallBack, gets server status updates.

class ServerDriver extends ServerStatusCallBack {
  Server server;

  @override
  updateServerStatus(Map<String, String> msg) =>
    msg.forEach((key, val) =>
      print('$key -- $val')); // peer specific message, where key of Map<String, String> is peerIP

  @override
  generalUpdate(String msg) => print(msg); // general status update about server

  init() {
    server = Server(
        '0.0.0.0',
        8000,
        <String>['192.168.1.103', '192.168.1.102'], // these are allowed peers( clients ), can request for fetching files
        <String>[
          '/path-to-file/image.png',
          '/path-to-file/spidy_love.zip',
        ],
        this); // this -- because ServerStatusCallBack is implemented in this class
  }

  start() {
    server.initServer(); // starts receiving incoming requests
  }
}

And finally main() function, which creates an instance of ServerDriver, starts server.

main() {
  var serverDriver = ServerDriver();
  serverDriver.init(); // binds server in a speficied address and port
  serverDriver.start(); // starts accepting incoming requests
}

Remember, this client-server program works in Local Area Network( LAN ). Interested in making it work in WAN, give it a try 👍.

Currently we've two working programs, a client and a server, written fully in Dart Language, which will be eventually put into a Flutter Application transferZ.

In next article of this series, we'll build Peer Discovery portion of transferZ.

In the mean time fork this repo, and try playing around.

You may consider following me on Twitter and GitHub, for more announcements.

See you soon 😉.