How can different types of data be transferred over a Network Socket?

#harmonyos #tcpsocket #networkkit #wearabledevice

Read the original article：How can different types of data be transferred over a Network Socket?

Requirement Description

Nowadays, network communication is utilized across a wide range of applications. Although it is often associated with IoT solutions, it can also be effectively applied in mobile and wearable applications to enable seamless data exchange and real-time interaction. For example, fitness trackers synchronize health data with smartphones in real time, navigation apps continuously update location information, and collaboration tools share files instantly across devices.

More specifically, the ability of devices connected to the same network to communicate with each other has become an indispensable technology in our daily lives. This is particularly evident in modern smart home environments, where smart speakers, lighting systems, thermostats, and security cameras interact seamlessly to provide users with a connected and automated living experience. Such communication not only enhances convenience but also improves efficiency, security, and overall user experience.

Background Knowledge

When developing HarmonyOS mobile and wearable applications, developers can leverage Network Kit to access a variety of networking capabilities that enable seamless communication and data transfer across devices. These capabilities go beyond simple protocol support, offering flexible options for establishing, managing, and optimizing network connections. The key features provided by Network Kit are as follows:

HTTP data request: initiates a data request through HTTP.
WebSocket connection: establishes a bidirectional connection between the server and client through WebSocket.
Socket connection: transmits data through Socket.
Network connection management: provides basic network management capabilities, including management of Wi-Fi/cellular/Ethernet connection priorities, network quality evaluation, subscription to network connection status changes, query of network connection information, and DNS resolution.
mDNS management: provides Multicast DNS (mDNS) management capabilities, such as adding, removing, discovering, and resolving local services on a LAN.

Implementation Steps

As an example, consider developing an application where a wearable device is used as a mouse for a Windows computer via a local network connection. In this scenario, the wearable device acts as the transmitter, while the computer serves as the receiver. The diagram below illustrates this setup;

In this context, two separate applications should be developed: one that acts as a transmitter for the wearable device, and another that acts as a receiver, collecting and processing data for the computer. It should be noted that both devices must be connected to the same network, as they will communicate via sockets. Below, you can see the Use Cases for the two different applications developed for each device.

For Wearable Device;

App Initialization
User Enters IP Address & Connects
TCP Socket Communication
Sensor Data Capture & Transmission
Cursor Position Calculation
Mouse Click Simulation
Sensor & Connection Management

For Computer;

Start the server and listen on the specified port.
Detect the local IP and display it to the user.
Wait for the client (watch) to connect and accept the connection when it arrives.
Receive incoming data as JSON and process it:
Handle connection timeout and errors.
Close any open connections when the server stops.

Code Snippets / Configuration

First, let's focus on the application developed for the wearable device, which connects via IP and functions like a mouse. It should be noted that the fundamental use case of the wearable application is to connect the user to the computer's receiver IP and transmit the sensor and click actions from the watch to the computer via this IP address. Accordingly, the Sensor Kit must be used to detect mouse movements. In the code block below, you can find all the related implementations, including the user interface.

import { sensor } from '@kit.SensorServiceKit';
import { BusinessError } from '@kit.BasicServicesKit';
import { promptAction } from '@kit.ArkUI';
import display from '@ohos.display';
import { socket } from '@kit.NetworkKit';

// Define interfaces for our data types
interface SensorData {
  type: string;
  x: number;
  y: number;
  z: number;
  timestamp: number;
}

interface ConfigData {
  type: string;
  sensitivity: number;
}

interface ClickData {
  type: string;
  button: string;
}

Imports the necessary HarmonyOS kits: SensorServiceKit for sensors, BasicServicesKit for error handling, ArkUI for UI interactions, display for screen info, and NetworkKit for TCP sockets.
Defines TypeScript interfaces for different types of data: sensor readings, configuration updates, and click events.

class SocketInfo {
  message: ArrayBuffer = new ArrayBuffer(1);
  remoteInfo: socket.SocketRemoteInfo = {} as socket.SocketRemoteInfo;
}

Defines a class to store TCP socket messages and remote client information.

@Entry
@Component
struct Index {
  @State accelX: string = '0.000';
  @State accelY: string = '0.000';
  @State accelZ: string = '0.000';
  @State gyroX: string = '0.000';
  @State gyroY: string = '0.000';
  @State gyroZ: string = '0.000';
  @State isMonitoring: boolean = false;
  @State isConnected: boolean = false;
  @State statusMessage: string = 'IP address';
  @State ipAddress: string = '';
  @State showConnectionPanel: boolean = true;

  @State cursorX: number = 111.5;
  @State cursorY: number = 111.5;
  @State circleColor: Color = Color.Red;
  @State leftClickActive: boolean = false;
  @State rightClickActive: boolean = false;

Defines the state variables for the watch UI, including sensor readings, connection status, cursor position, circle color, and click states.

  private screenWidth = display.getDefaultDisplaySync().width;
  private halfWidth = this.screenWidth / 2;

  private circleDiameter: number = 0
  private circleRadius: number = 0
  private circleCenterX: number = 0
  private circleCenterY: number = 0
  private dotRadius: number = 7.5;
  private sensitivity: number = 0.1;
  private edgeThreshold: number = 10;

Stores screen dimensions and calculates properties for the virtual mouse circle used in the UI.
sensitivity controls cursor speed, and edgeThreshold is used for proximity effects.

  // TCP Socket variables
  private tcp: socket.TCPSocket = socket.constructTCPSocketInstance();
  private remoteIP: string = '';
  private remotePort: number = 12345; // TARGET PORT

Initializes a TCP socket for communication with the computer.
Sets the default remote port to 12345.

  aboutToAppear(): void {
    this.circleDiameter = display.getDefaultDisplaySync().width/2
    this.circleRadius = this.circleDiameter / 2;
    this.circleCenterX = this.circleRadius;
    this.circleCenterY = this.circleRadius;

    this.setupSocketListeners();
  }

  aboutToDisappear(): void {
    this.stopSensors();
    this.closeConnection();
  }

aboutToAppear is called when the component is loaded: sets up circle dimensions and socket listeners.
aboutToDisappear is called when leaving the screen: stops sensors and closes the TCP connection.

private setupSocketListeners(): void {
    this.tcp.on('message', (value: SocketInfo) => {
      console.log("Message received");
      let buffer = value.message;
      let dataView = new DataView(buffer);
      let str = "";
      for (let i = 0; i < dataView.byteLength; ++i) {
        str += String.fromCharCode(dataView.getUint8(i));
      }
      console.log("Received message: " + str);
    });

    this.tcp.on('connect', () => {
      console.log("Connection established");
      this.isConnected = true;
      this.statusMessage = 'Connected! ✓';
      this.showConnectionPanel = false;
      promptAction.showToast({ message: 'Connected!', duration: 1000 });
      this.startSensors();
    });

    this.tcp.on('close', () => {
      console.log("Connection closed");
      this.isConnected = false;
      this.statusMessage = 'Connection Lost';
      this.showConnectionPanel = true;
    });
  }

Defines socket event listeners for:
- Receiving messages (message)
- Connection established (connect)
- Connection closed (close)
Updates UI states accordingly.

private connectToPC(): void {
    if (!this.ipAddress) {
      promptAction.showToast({ message: 'Please enter the IP Address', duration: 1000 });
      return;
    }

    this.remoteIP = this.ipAddress;
    this.statusMessage = 'Connecting...';

    try {
      // Bind local connection
      let localAddress: socket.NetAddress = {
        address: '0.0.0.0', // All network interfaces
        port: 0 // System will assign port automatically
      };

      this.tcp.bind(localAddress, (err: BusinessError) => {
        if (err) {
          console.error('Bind error: ' + JSON.stringify(err));
          this.statusMessage = 'Bind error';
          //return;
        }
        console.log('Bind successful');

        // Connect to remote computer
        let remoteAddress: socket.NetAddress = {
          address: this.remoteIP,
          port: this.remotePort
        };

        let connectOptions: socket.TCPConnectOptions = {
          address: remoteAddress,
          timeout: 5000
        };

        this.tcp.connect(connectOptions).then(() => {
          console.log('Connection successful');
          this.isConnected = true;
          this.statusMessage = 'Connected! ✓';
          this.showConnectionPanel = false;
        }).catch((err: BusinessError) => {
          console.error('Connection error: ' + JSON.stringify(err));
          this.statusMessage = 'Connection Error!';
          promptAction.showToast({ message: 'Connection Error', duration: 2000 });
        });
      });

    } catch (error) {
      console.error("Socket error: " + JSON.stringify(error));
      this.statusMessage = 'Socket Error';
    }
  }

Binds the socket to a local address.
Attempts to connect to the PC using the IP address provided by the user.
Updates connection status and handles errors.

private sendLeftClick(): void {
    if (!this.isConnected) return;

    const clickData: ClickData = {
      type: 'click',
      button: 'left',
    };
    this.sendDataToPC(clickData);
    this.leftClickActive = true;

    setTimeout(() => {
      this.leftClickActive = false;
    }, 300);
  }

  private sendRightClick(): void {
    if (!this.isConnected) return;

    const clickData: ClickData = {
      type: 'click',
      button: 'right',
    };
    this.sendDataToPC(clickData);
    this.rightClickActive = true;

    setTimeout(() => {
      this.rightClickActive = false;
    }, 300);
  }

  // Send data to computer with proper typing
  private sendDataToPC(data: SensorData | ConfigData | ClickData): void {
    if (this.isConnected) {
      try {
        const jsonData = JSON.stringify(data);
        let sendOptions: socket.TCPSendOptions = {
          data: jsonData
        };

        this.tcp.send(sendOptions).then(() => {
          // Data sent successfully
        }).catch((err: BusinessError) => {
          console.error('Send error: ' + JSON.stringify(err));
        });
      } catch (error) {
        console.error("Data send error: " + JSON.stringify(error));
      }
    }
  }

Sends left/right click events to the computer.
Uses sendDataToPC to send JSON-formatted data over the socket.

private closeConnection(): void {
    this.tcp.close().then(() => {
      console.log('Connection closed');
      this.isConnected = false;
      this.statusMessage = 'Connection Lost';
    }).catch((err: BusinessError) => {
      console.error('Close error: ' + JSON.stringify(err));
    });

    // Clean up event listeners
    this.tcp.off('message');
    this.tcp.off('connect');
    this.tcp.off('close');
  }

Closes the TCP connection and cleans up event listeners.
Updates the UI to reflect connection loss.

private startSensors(): void {
    try {
      sensor.on(sensor.SensorId.ACCELEROMETER, (data: sensor.AccelerometerResponse) => {
        this.accelX = data.x.toFixed(3);
        this.accelY = data.y.toFixed(3);
        this.accelZ = data.z.toFixed(3);

        this.updateCursorPosition(data.x, data.y);

        // Send accelerometer data to computer with proper type
        const sensorData: SensorData = {
          type: 'accel',
          x: data.x,
          y: data.y,
          z: data.z,
          timestamp: new Date().getTime()
        };
        this.sendDataToPC(sensorData);
      }, { interval: 50000 }); // 50ms

      sensor.on(sensor.SensorId.GYROSCOPE, (data: sensor.GyroscopeResponse) => {
        this.gyroX = data.x.toFixed(3);
        this.gyroY = data.y.toFixed(3);
        this.gyroZ = data.z.toFixed(3);

        // Send gyroscope data to computer with proper type
        const sensorData: SensorData = {
          type: 'gyro',
          x: data.x,
          y: data.y,
          z: data.z,
          timestamp: new Date().getTime()
        };
        this.sendDataToPC(sensorData);
      }, { interval: 50000 }); // 50ms

      this.isMonitoring = true;
      promptAction.showToast({ message: 'Sensors are started', duration: 1000 });
    } catch (error) {
      const e: BusinessError = error as BusinessError;
      console.error(`Sensor error. Code: ${e.code}, message: ${e.message}`);
      promptAction.showToast({ message: `Error: ${e.code}`, duration: 2000 });
    }
  }

  private updateCursorPosition(accelX: number, accelY: number): void {
    if (!this.isConnected) return;

    let newX = this.cursorX + (-accelX * this.sensitivity);
    let newY = this.cursorY + (accelY * this.sensitivity);

    const distanceFromCenter = Math.sqrt(
      Math.pow(newX + this.dotRadius - this.circleCenterX, 2) +
      Math.pow(newY + this.dotRadius - this.circleCenterY, 2)
    );

    if (distanceFromCenter > this.circleRadius - this.dotRadius) {
      const angle = Math.atan2(
        newY + this.dotRadius - this.circleCenterY,
        newX + this.dotRadius - this.circleCenterX
      );

      newX = this.circleCenterX + Math.cos(angle) * (this.circleRadius - this.dotRadius) - this.dotRadius;
      newY = this.circleCenterY + Math.sin(angle) * (this.circleRadius - this.dotRadius) - this.dotRadius;
    }

    this.checkEdgeProximity(distanceFromCenter);
    this.cursorX = newX;
    this.cursorY = newY;
  }

  private checkEdgeProximity(distanceFromCenter: number): void {
    const distanceToEdge = this.circleRadius - distanceFromCenter;

    if (distanceToEdge <= this.edgeThreshold) {
      this.circleColor = Color.Blue;
    } else {
      this.circleColor = Color.Red;
    }
  }

  private stopSensors(): void {
    try {
      sensor.off(sensor.SensorId.ACCELEROMETER);
      sensor.off(sensor.SensorId.GYROSCOPE);
      this.isMonitoring = false;
      promptAction.showToast({ message: 'Sensors are stopped', duration: 1000 });
    } catch (error) {
      const e: BusinessError = error as BusinessError;
      console.error(`Sensor stop error. Code: ${e.code}, message: ${e.message}`);
    }
  }

startSensors: registers accelerometer and gyroscope listeners.
updateCursorPosition: calculates the new cursor position based on sensor input.
checkEdgeProximity: changes circle color if cursor is near the edge.
stopSensors: unregisters sensor listeners.

  build() {
    Stack() { ... }
  }

Additionally, on the computer side, a driver needs to be developed and run to listen to and interpret the data coming from the wearable device. Accordingly, this driver listens to and processes the data over the specified IP. The corresponding code can be written in any language.