DEV Community: ying xu

A step-by-step guide to running Linux applications on your Android desktop

ying xu — Wed, 26 Aug 2026 07:17:12 +0000

As those who read our previous article likely know, OpenFDE is a Linux desktop environment built upon the AOSP graphics stack. While it is essentially an Android-based desktop, we have adapted the display and control mechanisms for Linux applications within the AOSP environment. This enables the seamless, integrated display of both Linux and Android applications, allowing users to run programs from different platforms—and leverage their respective software ecosystems—without even noticing the underlying differences. The result is OpenFDE: a Linux desktop environment that integrates seamlessly with Android.

Of course, this technology is not limited to the OpenFDE desktop; it can also be applied to other Android systems, such as smartphones and tablets. Let’s take a look at the underlying principles.

Explanation of Principles

Our concept for OpenFDE involves providing an entry point to launch Linux applications directly; therefore, after careful consideration, we initially adopted VNC technology to handle Linux input and output management.

VNC (Virtual Network Computing) is essentially a remote desktop control technology that enables users to access and control other computers remotely via the Internet or a local area network (LAN). It works by running a VNC server program on the remote computer; the user connects to this server using a VNC client program, allowing them to view and interact with the remote computer's desktop from their own machine.

The core principle involves the VNC app communicating with the VNC server via the RFB protocol; mouse and keyboard actions are transmitted to the VNC server as input. The VNC server utilizes the Xvnc module to implement the X protocol and communicate with Linux applications, while simultaneously forwarding text (in both Chinese and English) generated by the app to the IBus input method framework for processing (the specifics of the input method mechanism will be detailed later). Keyboard and mouse inputs are passed directly to the Linux application, which returns display data to the VNC server; finally, the display frames are sent back to the app as output.

It is worth noting that VNC relies on the RFB (Remote Frame Buffer) protocol for screen sharing between the client and server. It operates by rendering images locally and then transmitting them remotely to the client for decoding and display; this architecture ensures platform independence and cross-platform compatibility.

We selected this solution due to the technology's maturity and short development cycle, which allow for rapid requirement implementation and a product-grade user experience. However, there are drawbacks: VNC is primarily designed for remote control and imposes specific demands regarding efficiency and bandwidth (though latency remains within a few frames when running locally), and there are certain limitations regarding user interaction.

Let me now explain the key points in detail.

VNC server and client

Resulting Effect

When using the TigerVNC server, the actual parameters used to start vnc server after installation via apt are as follows:

/usr/bin/Xtigervnc :2 -desktop firefox_web_browser -auth /home/warlice/.Xauthority -geometry 1900x1200 -depth 24 -rfbwait 30000 -rfbport 5902 -pn -SecurityTypes None -BlacklistThreshold=10000000 -BlacklistTimeout=0

firefox_web_browser The script content is as follows:

#!/bin/bash
fde-set-ime-engine firefox_web_browser &   # Start input method, explained later
export GDK_BACKEND=x11
export QT_QPA_PLATFORM=xcb
export GTK_IM_MODULE=ibus
export QT_IM_MODULE=ibus
export QT4_IM_MODULE=ibus
export im=ibus
firefox &

The details regarding the input method are described later; here, the startup parameter configuration for vncserver specifies the layout dimensions (width and height), port, and connection timeout. Additionally, it specifies the startup program—Firefox—which is the application the user intends to launch and use. On OpenFDE, the implementation logic is as follows:

List all available programs on the Linux system and provide launch entry points.

When the user clicks the Linux program icon:

First, start the vncserver using the parameters mentioned above and launch the corresponding Linux application. Once startup is complete, open a new window within the FDE Android application and establish a VNC connection to display the frame data transmitted via the RFB protocol, while simultaneously sending input events—such as keyboard and mouse actions—to the server in accordance with the protocol.

This provides a seamless user experience, just like launching a Linux program directly on Android.

Source Code Solution

When chaining these functions together, modifications inevitably become necessary, requiring customization. After extensive comparative research, the project source code used is as follows.

APP: GitHub - iiordanov/remote-desktop-clients: VNC, RDP, SPICE, and oVirt/RHEV/Proxmox Clients for Android and Blackberry 10

This project involves the source code for bVNC, aRDP, aSPICE, and Opaque—four Android remote desktop clients. We are using the bVNC module; although there is a paid "Pro" version available, the documentation indicates that it merely adds encryption capabilities.

Server: GitHub - TigerVNC/tigervnc: High performance, multi-platform VNC client and server

Source code handling on the server side is somewhat more complex. The server must manage Linux applications and handle actual input and output—specifically, the graphical interfaces of these Linux programs. It utilizes the Xorg X server; VNC Server invokes Xorg via platform-specific libraries (such as Xvnc) and, following its own internal logic, converts the data into the RFB protocol for interaction with the client.

When compiling libraries on Linux that depend on xorg-xserver, you can compile x0vncserver if you only need to perform debugging.

x0vncserver - an inefficient VNC server which continuously polls any X

He cannot use all vncserver startup parameters; the result of launching it is a desktop output. You can refer toGitHub - TigerVNC/tigervnc: High performance, multi-platform VNC client and server to explore the implementation logic of the server.

Compiling TigerVNC Server

FDE uses the apt source code to modify and compile the TigerVNCServer deb package. The specific steps are as follows:

# Open /etc/apt/sources.list and uncomment all deb-src entries.
sudo apt update
sudo apt source tigervnc-standalone-server
// After entering the source code directory
sudo apt install equivs devscripts --no-install-recommends
sudo mk-build-deps -i -t "apt-get" -r
sudo DEB_BUILD_OPTIONS="parallel=8" dpkg-buildpackage -b -uc -us

The source code is managed using the quilt tool. For specific usage, please refer to:

[quilt Tool Usage Guide] f. quilt Tool Usage Guide

After successful compilation:

sudo dpkg -i  tigervnc-standalone-server_1.10.1+dfsg-3_arm64.deb

Input Processing and Input Methods

After starting the server and client and establishing a connection, communication follows the RFB protocol. The output is frame data, and what frame data to display is determined by the server side, which basically does not need to be modified. Most of the client's business logic is focused on handling input, i.e., mouse and keyboard events. The simplest approach is to pass all events directly to the server side in the activity's dispatchKeyEvent, which is how bVNC handles it.

    @Override
    public boolean dispatchKeyEvent(KeyEvent event) {
        return canvas.getKeyboard().keyEvent(event.getKeyCode(), event);
    }

Send data packets according to the message format:

Protocol message - mouse event message is as follows

  +--------------+--------------+--------------+
  | No. of bytes | Type [Value] | Description  |
  +--------------+--------------+--------------+
  | 1            | U8 [5]       | message-type |
  | 1            | U8           | button-mask  |
  | 2            | U16          | x-position   |
  | 2            | U16          | y-position   |
  +--------------+--------------+--------------+

Protocol message - keyboard event message is as follows

  +--------------+--------------+--------------+
  | No. of bytes | Type [Value] | Description  |
  +--------------+--------------+--------------+
  | 1            | U8 [4]       | message-type |
  | 1            | U8           | down-flag    |
  | 2            |              | padding      |
  | 4            | U32          | key          |
  +--------------+--------------+--------------+

For details, please refer to

[RFB Protocol Documentation] Input Protocol · RFB Remote Framebuffer Protocol

It should be noted whether the keyboard keysym used by Android is consistent with the keyboard layout recognized by Linux, that is, whether the keysym of sent events can be correctly recognized on the server side.

Input Method Requirements

In the use of VNC, there is one requirement that has never been well resolved: text input. English characters are relatively easy to handle—you can directly use key presses to output the desired characters. However, for Chinese characters and other characters in the unicode character set, Linux cannot directly generate them. It has been verified that Linux can receive a small number of Chinese characters in xkeysyms; you can bind a Chinese character to a keyboard key, but the number is also limited by the number of keyboard keys.

Therefore, if you want to input Chinese, the best approach is to add /usr/bin/ibus-daemon -d & to the startup script and then set the input method to the desired one. However, this also has a problem: the input method running on Linux is disconnected from the FDE desktop usage. How to handle the installation and uninstallation of input methods on Linux? Additionally, the display of the candidate word box is also an issue—if compatibility is not handled well, it will not be displayed in VNC.

For these reasons, FDE's VNC application integration solution adopts a more complete implementation. Modifications have been made to each stage of the input chain, though it ends at the ibus input method; X programs only serve as the text-receiving party.

The result is that using the Android input method on the Android VNC client side allows Chinese characters to be output to Linux programs opened on the server side.

This involves modifications in three stages:

The VNC APP can invoke the input method, intercept and forward characters generated by the input method to the vncserver, without affecting the original normal pathway;
The VNC server can forward received characters to the ibus input method, without affecting the original normal pathway;
The ibus input method can be normally invoked, receive character input into X programs, and does not affect the original normal input.

VNC APP

To implement input method invocation in the APP, EditText.request() suffices. However, to intercept and obtain the input method content, you need to override EditText, inject a custom InputConnection, and then obtain the input text through the commitText interface.

@Override
public boolean commitText(CharSequence text, int newCursorPosition) {
    if(DEBUG){
        Log.d(TAG, "commitText() called with: text = [" + text + "], mTextView = [" + detectEventEditText + "]");
    }
    if (detectEventEditText == null) {
        return super.commitText(text, newCursorPosition);
    }
    if (text instanceof Spanned) {
        Spanned spanned = (Spanned) text;
        SuggestionSpan[] spans = spanned.getSpans(0, text.length(), SuggestionSpan.class);
        Reflector.invokeMethodExceptionSafe(mIMM, "registerSuggestionSpansForNotification",
                new Reflector.TypedObject(spans, SuggestionSpan[].class));
    }
    Reflector.invokeMethodExceptionSafe(detectEventEditText, "resetErrorChangedFlag");
    Reflector.invokeMethodExceptionSafe(detectEventEditText, "hideErrorIfUnchanged");
    if(mInputModeFlag == INPUT_MODE_ONLY_KEYBOARD){
        return false;
    }
    if(!TextUtils.isEmpty(text)){
        detectEventEditText.getCanvas().getKeyboard().keyEvent(0xff, null, text.toString());
    }
    return true;
}

Actually, you could directly use com.android.internal.inputmethod.EditableInputConnection, but it is an internal final class. The author integrated it into the APP by copying the source code, and used reflection for some functions that cannot be called directly.

Key events not generated by the input method are sent directly via the RFB protocol by calling the interface in the activity.

@Override
public boolean dispatchKeyEvent(KeyEvent event) {
    return canvas.getKeyboard().keyEvent(event.getKeyCode(), event);
}

The difference is that characters generated by the input method use 0xff as the keycode, and are handled separately in RfbProto.

// Normal key press
public synchronized void writeKeyEvent(int keySym, int metaState, boolean down) {
    if (viewOnly)
        return;
    eventBufLen = 0;
    if (down) {
        writeModifierKeyEvents(metaState, down);
    }
    if (keySym > 0)
        writeKeyEvent(keySym, down);
    // Always release all modifiers after an "up" event
    if (!down) {
        writeModifierKeyEvents(metaState, down);
    }
    try {
        os.write(eventBuf, 0, eventBufLen);
    } catch (IOException e) {
        Log.e(TAG, "Failed to write key event to VNC server.");
        e.printStackTrace();
    }
}

// Input character
public synchronized void writeKeyStringEvent(int keycode, char c, boolean down) {
    if (viewOnly || os == null )
        return;
        int keySym = c;
        if (keySym > 0xff) {
            keySym += 0x1000000; //out of keyboard
        }
        Log.d(TAG, "writeKeyStringEvent() called with: keySym = [" + keySym + "], down = [" + down + "]");
        eventBufLen = 0;
        if (keySym > 0)
            writeSpacialKeyEvent(keycode, keySym, down);
        try {
            os.write(eventBuf, 0, eventBufLen);
        } catch (IOException e) {
            Log.e(TAG, "Failed to write key event to VNC server.");
            e.printStackTrace();
        }
}

See the code at:

bVNC/src/main/java/com/iiordanov/bVNC/DetectEventEditText.java · OpenFDE/remote-desktop-clients - Gitee

bVNC/src/main/java/com/iiordanov/bVNC/DetectInputConnection.java · OpenFDE/remote-desktop-clients - Gitee

bVNC/src/main/java/com/iiordanov/bVNC/RfbProto.java · OpenFDE/remote-desktop-clients - Gitee

VNC server

Upon receiving a KeyEvent from the VNC APP, according to the original protocol, the reserved bits are used as a flag to distinguish between input characters and key presses.

common/rfb/SMsgReader.cxx · OpenFDE/fde_tigervncserver - Gitee

void SMsgReader::readKeyEvent()
{
  // bool down = is->readU8();
  // is->skip(2);
  // rdr::U32 key = is->readU32();
  // handler->keyEvent(key, 0, down);
  bool down = is->readU8();
  rdr::U32 keycode = is->readU16();
//    is->skip(2); Use this reserved bit to distinguish between input characters and key presses 0xff
  rdr::U32 key = is->readU32();
  handler->keyEvent(key, keycode, down);
}

Other modifications on the server side use the quilt patch approach mentioned above; the main changes are all in this patch.

debian/patches/fde_patch_001.diff · OpenFDE/fde_tigervncserver - Gitee

The key points are as follows. Simply put, a 4-digit unicode is split into 4 key press events. For example, to send "我" (unicode "6211"), it would send "6", "2", "1", "1" as 4 down/up events respectively. If there is a "0", a special keysym needs to be sent; otherwise, the ibus protocol will convert it.

/*
 * vncKeyboardEvent() - add X11 events for the given RFB key event
 */
void vncKeyboardEvent(KeySym keysym, unsigned xtcode, int down)
{
    /* Simple case: the client has specified the key */
    //    xtcode = 0xff;
    if (xtcode == 0xff) {
        //hardcode keysym =  0xcccc , transfer IBUS_KEY_Adiaeresis maybe never used
        if (down) {
            vncKeysymKeyboardEvent(0xcccc, down);
            int fakeSym = (keysym >> 12) & 0xf;
            if(fakeSym == 0){
                fakeSendKeycode(0xcccd, down, xtcode);
                fakeSendKeycode(0xcccd, 0, xtcode);
            }else{
                fakeSendKeycode(fakeSym, down, xtcode);
                fakeSendKeycode(fakeSym, 0, xtcode);
            }
            fakeSym = (keysym >> 8) & 0xf;
            if(fakeSym == 0){
                fakeSendKeycode(0xcccd, down, xtcode);
                fakeSendKeycode(0xcccd, 0, xtcode);
            }else{
                fakeSendKeycode(fakeSym, down, xtcode);
                fakeSendKeycode(fakeSym, 0, xtcode);
            }
            fakeSym = (keysym >> 4) & 0xf;
            if(fakeSym == 0){
                fakeSendKeycode(0xcccd, down, xtcode);
                fakeSendKeycode(0xcccd, 0, xtcode);
            }else{
                fakeSendKeycode(fakeSym, down, xtcode);
                fakeSendKeycode(fakeSym, 0, xtcode);
            }
            fakeSym = keysym & 0xf;
            if(fakeSym == 0){
                fakeSendKeycode(0xcccd, down, xtcode);
                fakeSendKeycode(0xcccd, 0, xtcode);
            }else{
                fakeSendKeycode(fakeSym, down, xtcode);
                fakeSendKeycode(fakeSym, 0, xtcode);
            }
        } else {
            vncKeysymKeyboardEvent(0xcccc, down);
        }
    } else{
        fakeSendKeycode(keysym, down, xtcode);
    }
}

This can be understood as an encoding process. The reason this treatment is necessary is due to the ibus protocol transmission process: it seems that keysym types are only not escaped when they are 1 byte, or some defined in X11's keysymdef.h.

Ibus Input Method

Ultimately, text input still uses the input method on Linux. It is started via environment variables and serves no other purpose than to decode the content sent by vncserver and generate characters.

Simply follow the ibus input method flow and register a key callback:

    g_signal_connect(engine, "process-key-event", G_CALLBACK(engine_process_key_event_cb), NULL);

The keyval, keycode, and state sent by vncserver across 4 events are combined to generate a unicode character, which is then committed as text. Normal key presses do not have the flag bit 0xcccc used above, so they effectively bypass this input method.

gboolean engine_process_key_event_cb(IBusEngine *engine,
                                     guint keyval,
                                     guint keycode,
                                     guint state) {
    LOG_INFO("engine_process_key_event");
    ibus_engine_show_lookup_table(engine);
    // ibus_engine_show_preedit_text(engine);
    ibus_engine_show_auxiliary_text(engine);
    std::thread::id t1_id = std::this_thread::get_id();
    // char receiver[100]; //
    // sprintf(receiver, "\n\r    Received data  keycode:%d keyval:%d flag:%d text:%d ",keycode, keyval, flag, text);
    // engine_commit_text(engine, ibus_text_new_from_string(receiver));
    //1.unicode start
    if (keyval == 0xcccc && !(state & IBUS_RELEASE_MASK)) {
        // sprintf(receiver, "\n\r   1.unicode start   keycode:%d maks:%d  ",keycode, state & IBUS_RELEASE_MASK);
        // engine_commit_text(engine, ibus_text_new_from_string(receiver));
        flag = 3;
        text = 0;
        processing = 1;
        return TRUE;
    }
    //2.add unicode
    if (flag != -1 && !(state & IBUS_RELEASE_MASK)) {
        // sprintf(receiver, "\n    2.add unicode   flag:%d maks:%d  ",flag, state & IBUS_RELEASE_MASK);
        // engine_commit_text(engine, ibus_text_new_from_string(receiver));
        if(keyval == 0xcccd){
            text += (0 << (flag * 4));
        }else{
            text += (keyval << (flag * 4));
        }
        // char input_string[100]; //1
        // sprintf(input_string, "\n\r   Generate character  bit flag:%d keyval:%d text:%d",flag, keyval, text);
        // engine_commit_text(engine, ibus_text_new_from_string(input_string));
        flag--;
        return TRUE;
    }
    //3.unicode over
    if (keyval == 0xcccc && (state & IBUS_RELEASE_MASK)) {
        // sprintf(receiver, "\n\r    3.unicode over   keycode:%d maks:%d  flag:%d text:%d",keycode, state & IBUS_RELEASE_MASK, flag, text);
        // engine_commit_text(engine, ibus_text_new_from_string(receiver));
        flag = -1;
        wchar_t unicodeChar = static_cast<wchar_t>(text);
        std::wstring_convert <std::codecvt_utf8_utf16<wchar_t>> converter;
        std::string utf8Str = converter.to_bytes(unicodeChar);
        // char input_string[100]; //
        // sprintf(input_string, "\n\r     Key press keyval:%d keycode:%d text:%d  final====> ", keyval, keycode, text);
        // engine_commit_text(engine, ibus_text_new_from_string(input_string));
        engine_commit_text(engine, ibus_text_new_from_string(utf8Str.c_str()));
        processing = 0;
        return TRUE;
    }
    if (processing) {
        return TRUE;
    }
    text = 0;
    flag = -1;
    // sprintf(receiver, "\n\r     4.  maks:%d  " , state & IBUS_RELEASE_MASK);
    // engine_commit_text(engine, ibus_text_new_from_string(receiver));
    if (state & IBUS_RELEASE_MASK) {
        return FALSE;
    }
    return FALSE;
}

Clipboard

RFB has two versions of the clipboard protocol, Clipboard · RFB Remote Framebuffer Protocol

The old one does not support Chinese, so the extended clipboard pseudo-protocol needs to be used.

RFB 3.8 protocol limitation: the clipboard can only transmit the Latin-1 character set. In 2016, Cendio Ossman merged the Extended Clipboard Pseudo-Encoding into the main protocol branch, supporting transmission of the unicode character set in clipboard messages. UltraVNC/TigerVNC/RealVNC servers all support this extended protocol, while x11vnc has not yet provided support (2021/8/11).

The extended clipboard pseudo-protocol requires support from both client and server software. The message extends ServerCutText and ClientCutText, as follows:

  +--------------+--------------+--------------+
  | No. of bytes | Type [Value] | Description  |
  +--------------+--------------+--------------+
  | 1            | U8 [3/6]     | message-type |
  | 3            |              | padding      |
  | 4            | S32          | length       |
  | 4            | U32          | text-type    |
  | length-4     | U8 array     | text         |
  +--------------+--------------+--------------+

The remote-desktop-clients used in this project also does not implement this, VNC clipboard for utf8 text · Issue #285 · iiordanov/remote-desktop-clients.

The author implemented text copying through the extended clipboard pseudo-protocol. Since this protocol supports multiple formats, ZlibOutStream was used to convert the copied content. See the following patch for code modifications:

In fact, it can also implement file copying, but the author did not further implement it. remote-desktop-clients does not even implement Chinese (unicode) copying; the author speculates this is because clipboard usage scenarios on mobile are too rare, whereas OpenFDE has this requirement.

In summary, most of the principles of the VNC-based Linux application integration solution for the OpenFDE desktop environment are as described above. There are many more details to handle. It has already been released as a mature product, and we are subsequently researching methods to directly use xserver to run Linux programs within the OpenFDE desktop.

Future Plans

We are currently developing a new integration solution: porting xserver to our Android desktop. This way, starting an X Server within OpenFDE to connect to Linux programs allows direct transmission of the program's display content without going through screen image forwarding, greatly simplifying the flow. It offers better performance, compatibility, and user experience. At the same time, we will strive to achieve a window system that is as close as possible to that of Linux programs. Stay tuned.

That's all for this technical sharing session. Writing isn't easy, so if you found it helpful, please follow us for more~

How to Run Android Apps Natively on Linux?

ying xu — Mon, 27 Jul 2026 09:53:12 +0000

It is well known that the Android system runs on the Linux kernel . However, Android's rich and mature software ecosystem stands in stark contrast to that of Linux ; given that they share the same kernel, why can't Android applications run directly on Linux?

There are currently two main technical solutions for running Android on Linux: virtual machines and containerization . Virtual machines consume significant hardware resources and result in complete isolation—preventing interaction between the two systems—whereas containerization avoids these drawbacks, making it our preferred choice . Specifically, containerization effectively carves out a dedicated space for Android to run within the Linux environment while maintaining a channel for interaction between the two systems .

Therefore, based on this, we decided to create an open-source desktop, OpenFDE, which allows Android applications to run directly on Linux systems .

To provide a better understanding of our open-source project, this article outlines OpenFDE's software architecture and display logic .

1. OpenFDE Overall Software Architecture

┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                   Application Layer                                    │
│   ┌────────────────────────┬────────────────────────┐      ┌────────────┬──────────┐   │
│   │     Android App1       │      Android App2      │      │  X11 APP1  │ X11 APP1 │   │
│   └────────────────────────┴────────────────────────┘      └─────┬──────┴────┬─────┘   │
│                                                                  │           │         │
│                                                            ┌─────▼───────────▼──────┐  │
│                                                            │        XServer         │  │
│                                                            └────────────┬───────────┘  │
├─────────────────────────────────────────────────────────────────────────┼──────────────┤
│                                  API Framework Layer                    │              │
│   ┌────────────────────────┬────────────────────────┐  ┌────────────────▼───────────┐  │
│   │    Activity Manager    │     Window Manager     │  │   X11 Client (libX11, etc) │  │
│   ├────────────────────────┼────────────────────────┤  ├────────────────────────────┤  │
│   │    Content Providers   │      View System       │  │    DIX Driver (wiglamor)   │  │
│   ├────────────────────────┼────────────────────────┤  │            Glamor          │  │
│   │    Resource Manager    │    Location Manager    │  ├────────────────────────────┤  │
│   ├────────────────────────┴────────────────────────┤  │    EGL / GLX / OpenGL/ES   │  │
│   │             Android Runtime (ART)               │  ├────────────────────────────┤  │
│   └─────────────────────────────────────────────────┘  │   User-space Drivers       │  │
├────────────────────────────────────────────────────────┤   (libGL-Mesa-DRI, Mesa3D) │  │
│                                     Native Libs        │  ├────────────────────────────┤  │
│   ┌────────────────────────┬────────────────────────┐  │      LibDRM / GNU LibC     │  │
│   │          SSL           │        Graphics        │  └────────────────────────────┘  │
│   ├────────────────────────┴────────────────────────┤             ▲                    │
│   │   AOSP LibC with Modified Dynamic Libs Loader   │─────────────┘                    │
├─────────────────────────────────────────────────────┴──────────────────────────────────┤
│                           LXC Isolation and Fusion Layer                               │
│   ┌──────────────────┬──────────────────┬──────────────────┬──────────────────────┐    │
│   │ Filesystem Fusion│  Network Fusion  │  IPC Fusion &    │   Clipboard Fusion   │    │
│   │                  │                  │   Acceleration   │                      │    │
│   └──────────────────┴──────────────────┴──────────────────┴──────────────────────┘    │
├────────────────────────────────────────────────────────────────────────────────────────┤
│                           Standard Desktop Device Drivers                              │
├────────────────────────────────────────────────────────────────────────────────────────┤
│                                 Standard Linux Kernel                                  │
└────────────────────────────────────────────────────────────────────────────────────────┘

The image above illustrates our current overall technical architecture . From the bottom up, the architecture comprises :

Standard Linux Kernel
LXC Isolation and Integration Layer
API Framework Layer
Application Layer
Linux Desktop Graphics Stack

Let us examine each of these components:

Application Layer: The FDE application layer primarily encompasses Android applications and Linux X11 applications . While Android applications run natively, Linux X11 applications operate within the Linux system; we integrate their display into the Android environment using a VNC client running within FDE, while simultaneously forwarding input events (such as keyboard/mouse signals and IME inputs) . We are currently developing an Android-based X Server to replace the Linux-based VNC X Server, aiming to achieve superior display performance and enhanced compatibility .
API Framework Layer: It provides a set of standard API interfaces that enable applications to communicate and interact with the underlying system; these APIs cover functions such as accessing device drivers, managing file systems, and handling network communications . OpenFDE directly inherits the Android API design framework, ensuring that both adhere to a unified set of APIs for peripherals and power management . This significantly alleviates issues associated with the coexistence of different API generations, as well as the haphazard and chaotic nature of modifications during the usage and development of graphical interfaces .
LXC Isolation and Integration Layer: The LXC isolation layer isolates the Linux and Android file systems and API layers while sharing the underlying Linux kernel . The integration layer leverages standard desktop protocols (such as Wayland) and the Linux desktop graphics stack to provide key functions like graphical display, input device management, and window management . Additionally, we are working to package the Linux desktop graphics stack's OpenGL/ES components into dynamic libraries that can be called directly by Android, thereby ensuring compatibility with a wider range of hardware .

2. Display Integration Logic

Having covered the overall software architecture of OpenFDE, let us now discuss how the display component is integrated—a particularly challenging aspect of the entire integration solution .

Standard Linux DRI Display Framework

The logic of the DRI display framework on Linux (currently the most efficient display framework) is that the client directly invokes the GPU for rendering; once rendering is complete, the output is submitted to the compositor, which then performs composition and submits the result for display .

┌──────────────────┐    1. Render Request     ┌──────────────────┐
│ Client/App       ├─────────────────────────>│ GPU              │
└────────┬─────────┘                          └────────┬─────────┘
         │                                             │
         │ 2. Send Composition Request                 │ 3. Fetch Render Result
         ▼                                             ▼    for Composition
┌──────────────────┐                          ┌──────────────────┐
│ Compositor       ├─────────────────────────>│ DPU              │
└──────────────────┘    4. Send Composition   └──────────────────┘
                             Result

Native Android Display Logic

Android's display logic involves the application requesting buffers from SurfaceFlinger; SurfaceFlinger invokes Gralloc to fulfill the request and synchronizes the result for the application's use . The rendering process is executed directly by the application using the GPU, and the final output is handed over to SurfaceFlinger, which invokes HWC (Hardware Composer) to perform composition .

                 ┌──────────────────────────┐
                 │ Application (Self-Render)│<──────────┐
                 └─────┬──────────────┬─────┘           │
                       │              │                 │
  Submit Rendering     │              │ Apply Buffer    │ Return
  Results              │              │                 │
                       ▼              ▼                 │
┌─────────────┐   ┌─────────────────────────┐     ┌─────┴──────────┐
│ Display     │<──┤     surfaceflinger      ├────>│    gralloc     │
└──────▲──────┘   └───────────┬─────────────┘     └────────────────┘
       │                      │ Call
       │                      ▼
       │          ┌─────────────────────────┐
       └──────────┤  hwcomposer (Composition│
                  └─────────────────────────┘

OpenFDE Merged Display Architecture

Running Android on Linux allows these two workflows to be merged :

The initial stages follow the standard Android process .
hwcomposer is modified to act as a Wayland client (thereby losing hardware composition capabilities) .
When SurfaceFlinger requests composition from hwcomposer, the latter returns data indicating that the GPU should handle the entire composition task .
Consequently, SurfaceFlinger invokes the GPU to perform the composition and sends the result to hwcomposer for submission .
Finally, hwcomposer transmits the buffer via the Wayland protocol to the Linux compositor (mutter), which then sends it to the Linux DPU for display .

               ┌──────────────────────────┐
               │    App (Self-rendering)  │<──────────┐
               └─────┬──────────────┬─────┘           │
                     │              │                 │
Submit Rendering     │              │ Request Buffer  │ Allocate
Results              │              │                 │
                     ▼              ▼                 │
┌─────────────┐ Ask  ┌──────────────┴──────┐     ┌─────┴──────────┐
│ hwcomposer  │<────┤    surfaceflinger    ├────>│    gralloc     │
│ (wayland    │     └──────────────▲───────┘     └────────────────┘
│   client)   ├────────────────────┘
└──────┬──────┘ Respond: ALL DONE BY GPU
       │
       │ Transmit Buffer via Wayland
       ▼
┌─────────────┐
│   mutter    │
│(Composition)│
└──────┬──────┘
       │
       ▼
┌─────────────┐
│   Display   │
└─────────────┘

Of course, this approach entails some performance overhead, but it effectively achieves our objective .

3. Summary & What's Next

These are the two core components of how FDE currently operates . Beyond this, we have also worked on integrating Linux applications into Android, upgrading the UI, fine-tuning interactions, and more .

In the next installment, we'll dive into the technical principles behind FDE's Linux application integration .
"""