Seamless Inter-Process Communication with Godot's `execute_with_pipe`.

Jean Ekoé KOUSSAWO — Sat, 28 Dec 2024 11:21:25 +0000

Recently, I encountered some issues in Godot while attempting to communicate with another program through its stdin and stdout. After some days of research and testing various approaches, I finally understand what I believe to be the proper way to use OS.execute_with_pipe.

Some context

Why might you need to use OS.execute_with_pipe?

OS.execute_with_pipe lets you run external programs asynchronously and exchange data with them through input (stdin) and output (stdout) streams. See the official docs for more details.

You might need this when working with :

Command-line tools : Pinging a server, for instance,
AI models or scripts : Sending game state to an AI and receiving actions,
External program : Communicating with a custom application.

In my case, I encountered this while developing a game where players can train and use their own AI.

Working with stdin and stdout

OS.execute_with_pipe returns a dictionary containing:

"stdio": A FileAccess object for interacting with the process's stdin and stdout pipes (read/write),
"stderr": A FileAccess object for accessing the process's stderr pipe (read-only),
"pid": Process ID.

The one that interests us is "stdio" FileAccessobject. We can send data to the external process by writing to that file, and get input from the process by reading it.

Sending input to external program

Sending input is straightforward. You write data to the "stdio" object, ensuring that the input ends with a newline character (\n). This is analogous to pressing Enter on your keyboard. The easiest way to do this is with write_line():

# Example: sending input to the external process
process_io.write_line("This is my input") # Automatically adds \n

The external program will pause if it needs input and none is available. Inputs are handled in the order they are sent : First In, First Out (FIFO).

Receiving outputs from the external program

Retrieving output from the external program requires reading from "stdio".

Here was the pain for me.

Here's the catch: Read the stdout in a dedicated thread.

When you try to read stdout, if data is available in stdout, it’s retrieved immediately. However, if stdout is empty, the reading thread will pause until data arrives. Attempting this in the main thread risks freezing your application if the external process takes time to produce output.

Let me illustrate this.

Here is a link to the project I'll use for this.

Reading from main thread

Were are going to look at what is happening in main_thread_reading/MainTreadReading.tscn scene. The key element here is the main_thread_reading/MainTreadReading.gd script attached to the scene's root node. Additionally, the TextureRect node helps indicate if Godot is running smoothly: if it is, the Godot icon will fade in and out seamlessly.

Here is the script:

extends Control

var process_io : FileAccess
var process = null

func _ready():
    process = OS.execute_with_pipe("cmd.exe", ['/c', 'ping 8.8.8.8 -t'])
    process_io = process['stdio']


func _process(_delta: float) -> void:
    print(process_io.get_line())

This script executes a continuous ping request to 8.8.8.8 and prints the output line by line.

When you run the scene, you’ll notice that Godot occasionally freezes. This happens because the engine waits for output to be printed to the console, demonstrating why reading from stdout in the main thread can disrupt smooth execution.

A better way : reading from dedicated thread

Now let's look at the content of dedicated_thread_reading/DedicatedThreadReading.gd.

extends Control

var process_io : FileAccess
var process = null
var thread

func _ready():
    process = OS.execute_with_pipe("cmd.exe", ['/c', 'ping 8.8.8.8 -t'])
    process_io = process['stdio']
    thread = Thread.new()
    thread.start(read_process_output)

func read_process_output() :
    while process_io.is_open() and process_io.get_error() == OK :
        print(process_io.get_line())

In this version, we no longer read the output from the main thread. Instead, the output is read from a dedicated thread.

When you run the dedicated_thread_reading/DedicatedThreadReading.tscn scene, you’ll notice that Godot runs smoothly while continuously printing the ping request output to the console as it becomes available.

Conclusion

I hope this article has clarified how to use execute_with_pipe.

Before wrapping up, it’s worth noting that using threads in Godot can introduce its own challenges, particularly when accessing shared data or objects across multiple threads. To handle these situations safely, I recommend reading about Mutexes and Semaphores in the documentation.