DEV Community

Naman Vashistha
Naman Vashistha

Posted on

Implementing a Web-Based Terminal UI and Structured GET API Responses in LimeDB

#limedb #distributedsystems #keyvaluestore #apidesign

Github: namanvashistha/limedb

This commit introduces two significant enhancements to LimeDB: a web-based terminal user interface (TUI) and a more structured response for the /get API endpoint.

What Changed

  1. Web-Based Terminal UI: A new WebSocket server (tui/server.py) has been added to facilitate a web-based terminal experience. This server proxies input/output between a browser-based xterm.js client (tui/index.html) and the existing Python Textual TUI application (tui/main.py). The core of this functionality relies on pseudo-terminals (PTYs) to effectively tunnel the Textual TUI's standard I/O over WebSockets.

  2. Structured GET API Response: The /get/{key} endpoint in the NodeController and NodeService has been modified. Previously, it returned a raw String value. It now returns a GetResponse object, a newly introduced Java record (app/src/main/java/org/limedb/node/dto/GetResponse.java), which encapsulates both the retrieved value (String) and the nodeId (int) of the node that served the request.

Why the Change was Needed

  1. Web-Based TUI: The primary motivation for the web-based TUI is to enhance accessibility and ease of management for LimeDB clusters. By providing a browser-accessible terminal, operators can interact with the TUI from any location without needing to install specific terminal emulators or establish SSH connections directly to the nodes. This also opens avenues for easier integration into existing web-based monitoring or management dashboards.

  2. Structured GET API Response: In a distributed system like LimeDB, understanding the origin of data can be crucial for debugging, auditing, or performing intelligent client-side operations. The previous raw string response lacked this context. By returning a GetResponse object that includes the nodeId, API consumers gain immediate insight into which node handled the request, making the API more informative and robust for complex applications built on LimeDB.

Design Choices Made

  1. Web-Based TUI Architecture:

    • Client-Side: The tui/index.html leverages xterm.js for its robust terminal emulation capabilities within a web browser and xterm-addon-fit.js for dynamic resizing. This ensures a familiar and responsive terminal experience.
    • Server-Side: A Python asyncio and websockets based server (tui/server.py) acts as the intermediary. A key design decision was the use of a Pseudo-Terminal (PTY) via pty.openpty(). This allows the existing Textual TUI application (tui/main.py), which expects standard terminal I/O, to run as a child process. Its input and output are then piped through the PTY master, which the WebSocket server reads from and writes to. This approach minimizes changes to the Textual TUI application itself. Terminal resizing events from the browser are relayed to the PTY via fcntl.ioctl and termios.TIOCSWINSZ, allowing the Textual TUI to adapt its layout.

  2. Structured GET API Response:

    • A simple, immutable GetResponse Java record was chosen to encapsulate the value and nodeId. Using a record simplifies boilerplate code for DTOs. This approach ensures a clear, consistent data contract for clients.
    • The NodeController and NodeService were updated to consistently return ResponseEntity<GetResponse>, ensuring that both locally handled and peer-forwarded GET requests adhere to the new structured format. This centralizes the response formatting logic.

Trade-offs and Constraints

  1. Web-Based TUI:

    • Architectural Complexity: Introducing a separate Python-based WebSocket server and PTY bridging adds a new layer to the system's architecture, increasing deployment and operational overhead compared to a purely native TUI.
    • Performance/Latency: While generally acceptable for TUI interactions, the PTY I/O and WebSocket communication layers introduce marginal latency compared to direct native terminal usage.
    • Security: The current index.html connects to ws://localhost:8765, which is an unencrypted WebSocket. For production deployments, securing this connection with WSS (WebSocket Secure) and implementing robust authentication and authorization mechanisms would be critical.
    • Resource Consumption: Running the Python WebSocket server and a child Textual TUI process will consume additional CPU and memory resources on the node.

  2. Structured GET API Response:

    • API Backward Compatibility: This change is a breaking API modification. Existing clients that expect a raw string response from the /get endpoint will need to be updated to parse the new JSON GetResponse object.
    • Payload Size: The addition of the nodeId field and the overhead of JSON serialization slightly increase the response payload size. For applications with extremely small values and ultra-low latency requirements, this minor overhead could be a consideration, though for most key-value store use cases, the benefits of added context outweigh this.

Future Implications

  1. Web-Based TUI: This foundation paves the way for a more comprehensive web-based management and monitoring interface for LimeDB. Future work could include integrating multiple TUI instances, embedding operational metrics, or even providing a full graphical web console.

  2. Structured GET API Response: The adoption of a structured response for GET operations sets a precedent for enhancing other API endpoints with additional metadata. This pattern promotes API extensibility, allowing richer information to be conveyed to clients without tightly coupling the core data types, leading to a more robust and self-describing API over time.

Top comments (0)