DEV Community: 【OpenFDE】ying xu

Our Latest Technical Articles🎉

【OpenFDE】ying xu — Thu, 03 Sep 2026 10:05:56 +0000

【OpenFDE】ying xu

Sep 3

Break free from VNC and unlock a true multi-window Linux experience on Android.

#linux #ubuntu #android #software

5 min read

Break free from VNC and unlock a true multi-window Linux experience on Android.

【OpenFDE】ying xu — Thu, 03 Sep 2026 09:47:23 +0000

Effect Comparison：

Those who have read our previous technical article should already know that we used to run Linux applications on the desktop with the VNC solution，but this solution has a flaw：because it directly projects the Linux screen，there is always only one window during use，no matter how many pages you open，they are all confined within that frame，which is quite inconvenient.

So after research and exploration，we adopted a new technical solution—xserver，which can directly deliver program display content without screen image forwarding；the biggest change in practice is that you can open multiple windows，and the overall interaction is no different from Linux（for example，you can merge and split windows directly by dragging，and you are prompted whether to save when closing a document，etc.）。

Principle Explanation：

In case anyone does not know，before explaining the principle，let me mention that we are actually a Linux desktop based on AOSP，with the underlying layer based on Waydroid，so it is somewhat like an Android desktop that can run on Linux

This is a comparison of the technical structures of the two：you can see that VNC forwards operations and images through vncserver，while the xserver process is much simplified，performing better in aspects such as performance，compatibility，and user experience；next，we will explain the xserver solution in detail：

This is a detailed structure diagram of xserver. Let me first briefly introduce the background：Xserver is a core component of the X Window System，and X Window is actually one of the foundations of the graphical environments of Linux distributions；popular Linux desktop environments such as the well-known GNOME and KDE Plasma are all built on the X Window System. So what you can see is that we ported the xserver originally used in Linux systems to our FDE desktop，where it is responsible for managing display devices and processing requests from X clients，allowing Linux applications to run directly on our desktop through this approach.

Next，let us explain this structure in detail：the orange part on the left consists of the two modules responsible for display in Linux，which pass all display information，including window information，to the ported xserver；our Xserver runs in an Android service（XWindowService）in an independent process，and in this service native functions start a TCP server or a Unix socket server，so that Linux programs supporting the X11 protocol can connect to this server，send drawing instructions to it，and receive input events from it.

Many extensions and supplementary protocols on Xserver make it still the most compatible and efficient display solution on Linux，supporting both local and remote connections with very small memory usage. Compiling and debugging in the Android NDK is also quite convenient，with no redundant dependencies，and there are many mature solutions we can learn from.

Below XWindowService there is another module，which is WindowManager（window manager），a program used to manage and control X Window System (X11) windows. It handles operations such as opening，closing，moving，and resizing windows，and determines the appearance and behavior of windows. On OpenFDE，the Xserver connects to a custom-developed basic window manager，which runs in FDE-X11，has no window decoration function，and additionally implements functions such as a compositor，window property synchronization，and clipboard synchronization.

In the window manager，there is a relatively innovative point of ours called**Compositor（compositor）image redirection，**the Compositor extension is enabled in the window manager. Composite allows redirecting the output content of windows during the creation and display of all windows，redirecting the rendering of all windows in a window tree to internal storage，and then finally obtaining the image buffer of the window through the window pointer. The principle by which the compositor implements window effects is also based on this.

In this way，the image buffer of each window can be separated out and handed over to Android for drawing，that is，directly taking the image data from the Linux window into the Android window.

And the green part on the far right is SurfaceManager and Activity，responsible for displaying content and receiving events

Among them，SurfaceManager is a module abstracted out by FDE-X11，used to manage all Surfaces that draw buffers. Different Surfaces are passed into the EGL environment of Xserver so that different Surfaces correctly draw different buffers. For some OverrideRedirect windows，no new Surface will be created，and they will be drawn on the window they depend on.

As for the Activity part，it is mainly used as the window corresponding one-to-one with Linux. In the Android window system，the level handled by an APP window is confined to one level range，and only windows within this level range will have the same interaction behavior as windows of other APPs；currently，the only selectable Android components are Activity and Dialog. Their role is to create a SurfaceView to display the output image，receive Android input events，and send them to Xserver. Under the Android freeform mode used by FDE，the behavior is roughly equivalent to a Linux desktop，and in theory it can even be completely identical.

When a Linux window is created，an Activity is created in Android，and the image of the Linux window is displayed on the SurfaceView of this Activity；if the Activity receives keyboard or mouse input，it is sent to the corresponding coordinates of Xserver. There are many problems to handle in this process，such as lifecycle synchronization，window types，event redirection，window icon and title bar operations，and so on.

In addition to what is mentioned above，we also did related work such as Android and Linux window property synchronization，Android and Linux clipboard synchronization，which I will not elaborate on here.

In summary，unlike the current Xserver implementations on various platforms，because FDE-X11 itself is a desktop environment，with the power of AOSP and the ideas of some open-source solutions（such as Termux:X11），this innovative solution has been realized. The research and development of this solution took quite a long time，but there are still many immature aspects：for example，only part of the complex features of X windows have been implemented；in addition，Android windows are limited by the original design of the framework，and modifying them requires a huge amount of work，especially since the lifecycle overhead is one or two orders of magnitude higher than that of X windows；there is also the more efficient Wayland protocol. All of these leave a lot of room for optimizing user experience.

Hope this can give you all some reference and inspiration～The code of this solution is also placed in our code repository，and you can take it as you need. If you have any other questions，feel free to leave a message，and they will all be answered one by one.

A step-by-step guide to running Linux applications on your Android desktop

【OpenFDE】ying xu — Wed, 26 Aug 2026 07:17:12 +0000

As those who read our previous article likely know, OpenFDE is a Linux desktop environment built upon the AOSP graphics stack. While it is essentially an Android-based desktop, we have adapted the display and control mechanisms for Linux applications within the AOSP environment. This enables the seamless, integrated display of both Linux and Android applications, allowing users to run programs from different platforms—and leverage their respective software ecosystems—without even noticing the underlying differences. The result is OpenFDE: a Linux desktop environment that integrates seamlessly with Android.

Of course, this technology is not limited to the OpenFDE desktop; it can also be applied to other Android systems, such as smartphones and tablets. Let’s take a look at the underlying principles.

Explanation of Principles

Our concept for OpenFDE involves providing an entry point to launch Linux applications directly; therefore, after careful consideration, we initially adopted VNC technology to handle Linux input and output management.

VNC (Virtual Network Computing) is essentially a remote desktop control technology that enables users to access and control other computers remotely via the Internet or a local area network (LAN). It works by running a VNC server program on the remote computer; the user connects to this server using a VNC client program, allowing them to view and interact with the remote computer's desktop from their own machine.

The core principle involves the VNC app communicating with the VNC server via the RFB protocol; mouse and keyboard actions are transmitted to the VNC server as input. The VNC server utilizes the Xvnc module to implement the X protocol and communicate with Linux applications, while simultaneously forwarding text (in both Chinese and English) generated by the app to the IBus input method framework for processing (the specifics of the input method mechanism will be detailed later). Keyboard and mouse inputs are passed directly to the Linux application, which returns display data to the VNC server; finally, the display frames are sent back to the app as output.

It is worth noting that VNC relies on the RFB (Remote Frame Buffer) protocol for screen sharing between the client and server. It operates by rendering images locally and then transmitting them remotely to the client for decoding and display; this architecture ensures platform independence and cross-platform compatibility.

We selected this solution due to the technology's maturity and short development cycle, which allow for rapid requirement implementation and a product-grade user experience. However, there are drawbacks: VNC is primarily designed for remote control and imposes specific demands regarding efficiency and bandwidth (though latency remains within a few frames when running locally), and there are certain limitations regarding user interaction.

Let me now explain the key points in detail.

VNC server and client

Resulting Effect

When using the TigerVNC server, the actual parameters used to start vnc server after installation via apt are as follows:

/usr/bin/Xtigervnc :2 -desktop firefox_web_browser -auth /home/warlice/.Xauthority -geometry 1900x1200 -depth 24 -rfbwait 30000 -rfbport 5902 -pn -SecurityTypes None -BlacklistThreshold=10000000 -BlacklistTimeout=0

firefox_web_browser The script content is as follows:

#!/bin/bash
fde-set-ime-engine firefox_web_browser &   # Start input method, explained later
export GDK_BACKEND=x11
export QT_QPA_PLATFORM=xcb
export GTK_IM_MODULE=ibus
export QT_IM_MODULE=ibus
export QT4_IM_MODULE=ibus
export im=ibus
firefox &

The details regarding the input method are described later; here, the startup parameter configuration for vncserver specifies the layout dimensions (width and height), port, and connection timeout. Additionally, it specifies the startup program—Firefox—which is the application the user intends to launch and use. On OpenFDE, the implementation logic is as follows:

List all available programs on the Linux system and provide launch entry points.

When the user clicks the Linux program icon:

First, start the vncserver using the parameters mentioned above and launch the corresponding Linux application. Once startup is complete, open a new window within the FDE Android application and establish a VNC connection to display the frame data transmitted via the RFB protocol, while simultaneously sending input events—such as keyboard and mouse actions—to the server in accordance with the protocol.

This provides a seamless user experience, just like launching a Linux program directly on Android.

Source Code Solution

When chaining these functions together, modifications inevitably become necessary, requiring customization. After extensive comparative research, the project source code used is as follows.

APP: GitHub - iiordanov/remote-desktop-clients: VNC, RDP, SPICE, and oVirt/RHEV/Proxmox Clients for Android and Blackberry 10

This project involves the source code for bVNC, aRDP, aSPICE, and Opaque—four Android remote desktop clients. We are using the bVNC module; although there is a paid "Pro" version available, the documentation indicates that it merely adds encryption capabilities.

Server: GitHub - TigerVNC/tigervnc: High performance, multi-platform VNC client and server

Source code handling on the server side is somewhat more complex. The server must manage Linux applications and handle actual input and output—specifically, the graphical interfaces of these Linux programs. It utilizes the Xorg X server; VNC Server invokes Xorg via platform-specific libraries (such as Xvnc) and, following its own internal logic, converts the data into the RFB protocol for interaction with the client.

When compiling libraries on Linux that depend on xorg-xserver, you can compile x0vncserver if you only need to perform debugging.

x0vncserver - an inefficient VNC server which continuously polls any X

He cannot use all vncserver startup parameters; the result of launching it is a desktop output. You can refer toGitHub - TigerVNC/tigervnc: High performance, multi-platform VNC client and server to explore the implementation logic of the server.

Compiling TigerVNC Server

FDE uses the apt source code to modify and compile the TigerVNCServer deb package. The specific steps are as follows:

# Open /etc/apt/sources.list and uncomment all deb-src entries.
sudo apt update
sudo apt source tigervnc-standalone-server
// After entering the source code directory
sudo apt install equivs devscripts --no-install-recommends
sudo mk-build-deps -i -t "apt-get" -r
sudo DEB_BUILD_OPTIONS="parallel=8" dpkg-buildpackage -b -uc -us

The source code is managed using the quilt tool. For specific usage, please refer to:

[quilt Tool Usage Guide] f. quilt Tool Usage Guide

After successful compilation:

sudo dpkg -i  tigervnc-standalone-server_1.10.1+dfsg-3_arm64.deb

Input Processing and Input Methods

After starting the server and client and establishing a connection, communication follows the RFB protocol. The output is frame data, and what frame data to display is determined by the server side, which basically does not need to be modified. Most of the client's business logic is focused on handling input, i.e., mouse and keyboard events. The simplest approach is to pass all events directly to the server side in the activity's dispatchKeyEvent, which is how bVNC handles it.

    @Override
    public boolean dispatchKeyEvent(KeyEvent event) {
        return canvas.getKeyboard().keyEvent(event.getKeyCode(), event);
    }

Send data packets according to the message format:

Protocol message - mouse event message is as follows

  +--------------+--------------+--------------+
  | No. of bytes | Type [Value] | Description  |
  +--------------+--------------+--------------+
  | 1            | U8 [5]       | message-type |
  | 1            | U8           | button-mask  |
  | 2            | U16          | x-position   |
  | 2            | U16          | y-position   |
  +--------------+--------------+--------------+

Protocol message - keyboard event message is as follows

  +--------------+--------------+--------------+
  | No. of bytes | Type [Value] | Description  |
  +--------------+--------------+--------------+
  | 1            | U8 [4]       | message-type |
  | 1            | U8           | down-flag    |
  | 2            |              | padding      |
  | 4            | U32          | key          |
  +--------------+--------------+--------------+

For details, please refer to

[RFB Protocol Documentation] Input Protocol · RFB Remote Framebuffer Protocol

It should be noted whether the keyboard keysym used by Android is consistent with the keyboard layout recognized by Linux, that is, whether the keysym of sent events can be correctly recognized on the server side.

Input Method Requirements

In the use of VNC, there is one requirement that has never been well resolved: text input. English characters are relatively easy to handle—you can directly use key presses to output the desired characters. However, for Chinese characters and other characters in the unicode character set, Linux cannot directly generate them. It has been verified that Linux can receive a small number of Chinese characters in xkeysyms; you can bind a Chinese character to a keyboard key, but the number is also limited by the number of keyboard keys.

Therefore, if you want to input Chinese, the best approach is to add /usr/bin/ibus-daemon -d & to the startup script and then set the input method to the desired one. However, this also has a problem: the input method running on Linux is disconnected from the FDE desktop usage. How to handle the installation and uninstallation of input methods on Linux? Additionally, the display of the candidate word box is also an issue—if compatibility is not handled well, it will not be displayed in VNC.

For these reasons, FDE's VNC application integration solution adopts a more complete implementation. Modifications have been made to each stage of the input chain, though it ends at the ibus input method; X programs only serve as the text-receiving party.

The result is that using the Android input method on the Android VNC client side allows Chinese characters to be output to Linux programs opened on the server side.

This involves modifications in three stages:

The VNC APP can invoke the input method, intercept and forward characters generated by the input method to the vncserver, without affecting the original normal pathway;
The VNC server can forward received characters to the ibus input method, without affecting the original normal pathway;
The ibus input method can be normally invoked, receive character input into X programs, and does not affect the original normal input.

VNC APP

To implement input method invocation in the APP, EditText.request() suffices. However, to intercept and obtain the input method content, you need to override EditText, inject a custom InputConnection, and then obtain the input text through the commitText interface.

@Override
public boolean commitText(CharSequence text, int newCursorPosition) {
    if(DEBUG){
        Log.d(TAG, "commitText() called with: text = [" + text + "], mTextView = [" + detectEventEditText + "]");
    }
    if (detectEventEditText == null) {
        return super.commitText(text, newCursorPosition);
    }
    if (text instanceof Spanned) {
        Spanned spanned = (Spanned) text;
        SuggestionSpan[] spans = spanned.getSpans(0, text.length(), SuggestionSpan.class);
        Reflector.invokeMethodExceptionSafe(mIMM, "registerSuggestionSpansForNotification",
                new Reflector.TypedObject(spans, SuggestionSpan[].class));
    }
    Reflector.invokeMethodExceptionSafe(detectEventEditText, "resetErrorChangedFlag");
    Reflector.invokeMethodExceptionSafe(detectEventEditText, "hideErrorIfUnchanged");
    if(mInputModeFlag == INPUT_MODE_ONLY_KEYBOARD){
        return false;
    }
    if(!TextUtils.isEmpty(text)){
        detectEventEditText.getCanvas().getKeyboard().keyEvent(0xff, null, text.toString());
    }
    return true;
}

Actually, you could directly use com.android.internal.inputmethod.EditableInputConnection, but it is an internal final class. The author integrated it into the APP by copying the source code, and used reflection for some functions that cannot be called directly.

Key events not generated by the input method are sent directly via the RFB protocol by calling the interface in the activity.

@Override
public boolean dispatchKeyEvent(KeyEvent event) {
    return canvas.getKeyboard().keyEvent(event.getKeyCode(), event);
}

The difference is that characters generated by the input method use 0xff as the keycode, and are handled separately in RfbProto.

// Normal key press
public synchronized void writeKeyEvent(int keySym, int metaState, boolean down) {
    if (viewOnly)
        return;
    eventBufLen = 0;
    if (down) {
        writeModifierKeyEvents(metaState, down);
    }
    if (keySym > 0)
        writeKeyEvent(keySym, down);
    // Always release all modifiers after an "up" event
    if (!down) {
        writeModifierKeyEvents(metaState, down);
    }
    try {
        os.write(eventBuf, 0, eventBufLen);
    } catch (IOException e) {
        Log.e(TAG, "Failed to write key event to VNC server.");
        e.printStackTrace();
    }
}

// Input character
public synchronized void writeKeyStringEvent(int keycode, char c, boolean down) {
    if (viewOnly || os == null )
        return;
        int keySym = c;
        if (keySym > 0xff) {
            keySym += 0x1000000; //out of keyboard
        }
        Log.d(TAG, "writeKeyStringEvent() called with: keySym = [" + keySym + "], down = [" + down + "]");
        eventBufLen = 0;
        if (keySym > 0)
            writeSpacialKeyEvent(keycode, keySym, down);
        try {
            os.write(eventBuf, 0, eventBufLen);
        } catch (IOException e) {
            Log.e(TAG, "Failed to write key event to VNC server.");
            e.printStackTrace();
        }
}

See the code at:

bVNC/src/main/java/com/iiordanov/bVNC/DetectEventEditText.java · OpenFDE/remote-desktop-clients - Gitee

bVNC/src/main/java/com/iiordanov/bVNC/DetectInputConnection.java · OpenFDE/remote-desktop-clients - Gitee

bVNC/src/main/java/com/iiordanov/bVNC/RfbProto.java · OpenFDE/remote-desktop-clients - Gitee

VNC server

Upon receiving a KeyEvent from the VNC APP, according to the original protocol, the reserved bits are used as a flag to distinguish between input characters and key presses.

common/rfb/SMsgReader.cxx · OpenFDE/fde_tigervncserver - Gitee

void SMsgReader::readKeyEvent()
{
  // bool down = is->readU8();
  // is->skip(2);
  // rdr::U32 key = is->readU32();
  // handler->keyEvent(key, 0, down);
  bool down = is->readU8();
  rdr::U32 keycode = is->readU16();
//    is->skip(2); Use this reserved bit to distinguish between input characters and key presses 0xff
  rdr::U32 key = is->readU32();
  handler->keyEvent(key, keycode, down);
}

Other modifications on the server side use the quilt patch approach mentioned above; the main changes are all in this patch.

debian/patches/fde_patch_001.diff · OpenFDE/fde_tigervncserver - Gitee

The key points are as follows. Simply put, a 4-digit unicode is split into 4 key press events. For example, to send "我" (unicode "6211"), it would send "6", "2", "1", "1" as 4 down/up events respectively. If there is a "0", a special keysym needs to be sent; otherwise, the ibus protocol will convert it.

/*
 * vncKeyboardEvent() - add X11 events for the given RFB key event
 */
void vncKeyboardEvent(KeySym keysym, unsigned xtcode, int down)
{
    /* Simple case: the client has specified the key */
    //    xtcode = 0xff;
    if (xtcode == 0xff) {
        //hardcode keysym =  0xcccc , transfer IBUS_KEY_Adiaeresis maybe never used
        if (down) {
            vncKeysymKeyboardEvent(0xcccc, down);
            int fakeSym = (keysym >> 12) & 0xf;
            if(fakeSym == 0){
                fakeSendKeycode(0xcccd, down, xtcode);
                fakeSendKeycode(0xcccd, 0, xtcode);
            }else{
                fakeSendKeycode(fakeSym, down, xtcode);
                fakeSendKeycode(fakeSym, 0, xtcode);
            }
            fakeSym = (keysym >> 8) & 0xf;
            if(fakeSym == 0){
                fakeSendKeycode(0xcccd, down, xtcode);
                fakeSendKeycode(0xcccd, 0, xtcode);
            }else{
                fakeSendKeycode(fakeSym, down, xtcode);
                fakeSendKeycode(fakeSym, 0, xtcode);
            }
            fakeSym = (keysym >> 4) & 0xf;
            if(fakeSym == 0){
                fakeSendKeycode(0xcccd, down, xtcode);
                fakeSendKeycode(0xcccd, 0, xtcode);
            }else{
                fakeSendKeycode(fakeSym, down, xtcode);
                fakeSendKeycode(fakeSym, 0, xtcode);
            }
            fakeSym = keysym & 0xf;
            if(fakeSym == 0){
                fakeSendKeycode(0xcccd, down, xtcode);
                fakeSendKeycode(0xcccd, 0, xtcode);
            }else{
                fakeSendKeycode(fakeSym, down, xtcode);
                fakeSendKeycode(fakeSym, 0, xtcode);
            }
        } else {
            vncKeysymKeyboardEvent(0xcccc, down);
        }
    } else{
        fakeSendKeycode(keysym, down, xtcode);
    }
}

This can be understood as an encoding process. The reason this treatment is necessary is due to the ibus protocol transmission process: it seems that keysym types are only not escaped when they are 1 byte, or some defined in X11's keysymdef.h.

Ibus Input Method

Ultimately, text input still uses the input method on Linux. It is started via environment variables and serves no other purpose than to decode the content sent by vncserver and generate characters.

Simply follow the ibus input method flow and register a key callback:

    g_signal_connect(engine, "process-key-event", G_CALLBACK(engine_process_key_event_cb), NULL);

The keyval, keycode, and state sent by vncserver across 4 events are combined to generate a unicode character, which is then committed as text. Normal key presses do not have the flag bit 0xcccc used above, so they effectively bypass this input method.

gboolean engine_process_key_event_cb(IBusEngine *engine,
                                     guint keyval,
                                     guint keycode,
                                     guint state) {
    LOG_INFO("engine_process_key_event");
    ibus_engine_show_lookup_table(engine);
    // ibus_engine_show_preedit_text(engine);
    ibus_engine_show_auxiliary_text(engine);
    std::thread::id t1_id = std::this_thread::get_id();
    // char receiver[100]; //
    // sprintf(receiver, "\n\r    Received data  keycode:%d keyval:%d flag:%d text:%d ",keycode, keyval, flag, text);
    // engine_commit_text(engine, ibus_text_new_from_string(receiver));
    //1.unicode start
    if (keyval == 0xcccc && !(state & IBUS_RELEASE_MASK)) {
        // sprintf(receiver, "\n\r   1.unicode start   keycode:%d maks:%d  ",keycode, state & IBUS_RELEASE_MASK);
        // engine_commit_text(engine, ibus_text_new_from_string(receiver));
        flag = 3;
        text = 0;
        processing = 1;
        return TRUE;
    }
    //2.add unicode
    if (flag != -1 && !(state & IBUS_RELEASE_MASK)) {
        // sprintf(receiver, "\n    2.add unicode   flag:%d maks:%d  ",flag, state & IBUS_RELEASE_MASK);
        // engine_commit_text(engine, ibus_text_new_from_string(receiver));
        if(keyval == 0xcccd){
            text += (0 << (flag * 4));
        }else{
            text += (keyval << (flag * 4));
        }
        // char input_string[100]; //1
        // sprintf(input_string, "\n\r   Generate character  bit flag:%d keyval:%d text:%d",flag, keyval, text);
        // engine_commit_text(engine, ibus_text_new_from_string(input_string));
        flag--;
        return TRUE;
    }
    //3.unicode over
    if (keyval == 0xcccc && (state & IBUS_RELEASE_MASK)) {
        // sprintf(receiver, "\n\r    3.unicode over   keycode:%d maks:%d  flag:%d text:%d",keycode, state & IBUS_RELEASE_MASK, flag, text);
        // engine_commit_text(engine, ibus_text_new_from_string(receiver));
        flag = -1;
        wchar_t unicodeChar = static_cast<wchar_t>(text);
        std::wstring_convert <std::codecvt_utf8_utf16<wchar_t>> converter;
        std::string utf8Str = converter.to_bytes(unicodeChar);
        // char input_string[100]; //
        // sprintf(input_string, "\n\r     Key press keyval:%d keycode:%d text:%d  final====> ", keyval, keycode, text);
        // engine_commit_text(engine, ibus_text_new_from_string(input_string));
        engine_commit_text(engine, ibus_text_new_from_string(utf8Str.c_str()));
        processing = 0;
        return TRUE;
    }
    if (processing) {
        return TRUE;
    }
    text = 0;
    flag = -1;
    // sprintf(receiver, "\n\r     4.  maks:%d  " , state & IBUS_RELEASE_MASK);
    // engine_commit_text(engine, ibus_text_new_from_string(receiver));
    if (state & IBUS_RELEASE_MASK) {
        return FALSE;
    }
    return FALSE;
}

Clipboard

RFB has two versions of the clipboard protocol, Clipboard · RFB Remote Framebuffer Protocol

The old one does not support Chinese, so the extended clipboard pseudo-protocol needs to be used.

RFB 3.8 protocol limitation: the clipboard can only transmit the Latin-1 character set. In 2016, Cendio Ossman merged the Extended Clipboard Pseudo-Encoding into the main protocol branch, supporting transmission of the unicode character set in clipboard messages. UltraVNC/TigerVNC/RealVNC servers all support this extended protocol, while x11vnc has not yet provided support (2021/8/11).

The extended clipboard pseudo-protocol requires support from both client and server software. The message extends ServerCutText and ClientCutText, as follows:

  +--------------+--------------+--------------+
  | No. of bytes | Type [Value] | Description  |
  +--------------+--------------+--------------+
  | 1            | U8 [3/6]     | message-type |
  | 3            |              | padding      |
  | 4            | S32          | length       |
  | 4            | U32          | text-type    |
  | length-4     | U8 array     | text         |
  +--------------+--------------+--------------+

The remote-desktop-clients used in this project also does not implement this, VNC clipboard for utf8 text · Issue #285 · iiordanov/remote-desktop-clients.

The author implemented text copying through the extended clipboard pseudo-protocol. Since this protocol supports multiple formats, ZlibOutStream was used to convert the copied content. See the following patch for code modifications:

In fact, it can also implement file copying, but the author did not further implement it. remote-desktop-clients does not even implement Chinese (unicode) copying; the author speculates this is because clipboard usage scenarios on mobile are too rare, whereas OpenFDE has this requirement.

In summary, most of the principles of the VNC-based Linux application integration solution for the OpenFDE desktop environment are as described above. There are many more details to handle. It has already been released as a mature product, and we are subsequently researching methods to directly use xserver to run Linux programs within the OpenFDE desktop.

Future Plans

We are currently developing a new integration solution: porting xserver to our Android desktop. This way, starting an X Server within OpenFDE to connect to Linux programs allows direct transmission of the program's display content without going through screen image forwarding, greatly simplifying the flow. It offers better performance, compatibility, and user experience. At the same time, we will strive to achieve a window system that is as close as possible to that of Linux programs. Stay tuned.

That's all for this technical sharing session. Writing isn't easy, so if you found it helpful, please follow us for more~

How to Run Android Apps Natively on Linux?

【OpenFDE】ying xu — Mon, 27 Jul 2026 09:53:12 +0000

It is well known that the Android system runs on the Linux kernel . However, Android's rich and mature software ecosystem stands in stark contrast to that of Linux ; given that they share the same kernel, why can't Android applications run directly on Linux?

There are currently two main technical solutions for running Android on Linux: virtual machines and containerization . Virtual machines consume significant hardware resources and result in complete isolation—preventing interaction between the two systems—whereas containerization avoids these drawbacks, making it our preferred choice . Specifically, containerization effectively carves out a dedicated space for Android to run within the Linux environment while maintaining a channel for interaction between the two systems .

Therefore, based on this, we decided to create an open-source desktop, OpenFDE, which allows Android applications to run directly on Linux systems .

To provide a better understanding of our open-source project, this article outlines OpenFDE's software architecture and display logic .

1. OpenFDE Overall Software Architecture

┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                   Application Layer                                    │
│   ┌────────────────────────┬────────────────────────┐      ┌────────────┬──────────┐   │
│   │     Android App1       │      Android App2      │      │  X11 APP1  │ X11 APP1 │   │
│   └────────────────────────┴────────────────────────┘      └─────┬──────┴────┬─────┘   │
│                                                                  │           │         │
│                                                            ┌─────▼───────────▼──────┐  │
│                                                            │        XServer         │  │
│                                                            └────────────┬───────────┘  │
├─────────────────────────────────────────────────────────────────────────┼──────────────┤
│                                  API Framework Layer                    │              │
│   ┌────────────────────────┬────────────────────────┐  ┌────────────────▼───────────┐  │
│   │    Activity Manager    │     Window Manager     │  │   X11 Client (libX11, etc) │  │
│   ├────────────────────────┼────────────────────────┤  ├────────────────────────────┤  │
│   │    Content Providers   │      View System       │  │    DIX Driver (wiglamor)   │  │
│   ├────────────────────────┼────────────────────────┤  │            Glamor          │  │
│   │    Resource Manager    │    Location Manager    │  ├────────────────────────────┤  │
│   ├────────────────────────┴────────────────────────┤  │    EGL / GLX / OpenGL/ES   │  │
│   │             Android Runtime (ART)               │  ├────────────────────────────┤  │
│   └─────────────────────────────────────────────────┘  │   User-space Drivers       │  │
├────────────────────────────────────────────────────────┤   (libGL-Mesa-DRI, Mesa3D) │  │
│                                     Native Libs        │  ├────────────────────────────┤  │
│   ┌────────────────────────┬────────────────────────┐  │      LibDRM / GNU LibC     │  │
│   │          SSL           │        Graphics        │  └────────────────────────────┘  │
│   ├────────────────────────┴────────────────────────┤             ▲                    │
│   │   AOSP LibC with Modified Dynamic Libs Loader   │─────────────┘                    │
├─────────────────────────────────────────────────────┴──────────────────────────────────┤
│                           LXC Isolation and Fusion Layer                               │
│   ┌──────────────────┬──────────────────┬──────────────────┬──────────────────────┐    │
│   │ Filesystem Fusion│  Network Fusion  │  IPC Fusion &    │   Clipboard Fusion   │    │
│   │                  │                  │   Acceleration   │                      │    │
│   └──────────────────┴──────────────────┴──────────────────┴──────────────────────┘    │
├────────────────────────────────────────────────────────────────────────────────────────┤
│                           Standard Desktop Device Drivers                              │
├────────────────────────────────────────────────────────────────────────────────────────┤
│                                 Standard Linux Kernel                                  │
└────────────────────────────────────────────────────────────────────────────────────────┘

The image above illustrates our current overall technical architecture . From the bottom up, the architecture comprises :

Standard Linux Kernel
LXC Isolation and Integration Layer
API Framework Layer
Application Layer
Linux Desktop Graphics Stack

Let us examine each of these components:

Application Layer: The FDE application layer primarily encompasses Android applications and Linux X11 applications . While Android applications run natively, Linux X11 applications operate within the Linux system; we integrate their display into the Android environment using a VNC client running within FDE, while simultaneously forwarding input events (such as keyboard/mouse signals and IME inputs) . We are currently developing an Android-based X Server to replace the Linux-based VNC X Server, aiming to achieve superior display performance and enhanced compatibility .
API Framework Layer: It provides a set of standard API interfaces that enable applications to communicate and interact with the underlying system; these APIs cover functions such as accessing device drivers, managing file systems, and handling network communications . OpenFDE directly inherits the Android API design framework, ensuring that both adhere to a unified set of APIs for peripherals and power management . This significantly alleviates issues associated with the coexistence of different API generations, as well as the haphazard and chaotic nature of modifications during the usage and development of graphical interfaces .
LXC Isolation and Integration Layer: The LXC isolation layer isolates the Linux and Android file systems and API layers while sharing the underlying Linux kernel . The integration layer leverages standard desktop protocols (such as Wayland) and the Linux desktop graphics stack to provide key functions like graphical display, input device management, and window management . Additionally, we are working to package the Linux desktop graphics stack's OpenGL/ES components into dynamic libraries that can be called directly by Android, thereby ensuring compatibility with a wider range of hardware .

2. Display Integration Logic

Having covered the overall software architecture of OpenFDE, let us now discuss how the display component is integrated—a particularly challenging aspect of the entire integration solution .

Standard Linux DRI Display Framework

The logic of the DRI display framework on Linux (currently the most efficient display framework) is that the client directly invokes the GPU for rendering; once rendering is complete, the output is submitted to the compositor, which then performs composition and submits the result for display .

┌──────────────────┐    1. Render Request     ┌──────────────────┐
│ Client/App       ├─────────────────────────>│ GPU              │
└────────┬─────────┘                          └────────┬─────────┘
         │                                             │
         │ 2. Send Composition Request                 │ 3. Fetch Render Result
         ▼                                             ▼    for Composition
┌──────────────────┐                          ┌──────────────────┐
│ Compositor       ├─────────────────────────>│ DPU              │
└──────────────────┘    4. Send Composition   └──────────────────┘
                             Result

Native Android Display Logic

Android's display logic involves the application requesting buffers from SurfaceFlinger; SurfaceFlinger invokes Gralloc to fulfill the request and synchronizes the result for the application's use . The rendering process is executed directly by the application using the GPU, and the final output is handed over to SurfaceFlinger, which invokes HWC (Hardware Composer) to perform composition .

                 ┌──────────────────────────┐
                 │ Application (Self-Render)│<──────────┐
                 └─────┬──────────────┬─────┘           │
                       │              │                 │
  Submit Rendering     │              │ Apply Buffer    │ Return
  Results              │              │                 │
                       ▼              ▼                 │
┌─────────────┐   ┌─────────────────────────┐     ┌─────┴──────────┐
│ Display     │<──┤     surfaceflinger      ├────>│    gralloc     │
└──────▲──────┘   └───────────┬─────────────┘     └────────────────┘
       │                      │ Call
       │                      ▼
       │          ┌─────────────────────────┐
       └──────────┤  hwcomposer (Composition│
                  └─────────────────────────┘

OpenFDE Merged Display Architecture

Running Android on Linux allows these two workflows to be merged :

The initial stages follow the standard Android process .
hwcomposer is modified to act as a Wayland client (thereby losing hardware composition capabilities) .
When SurfaceFlinger requests composition from hwcomposer, the latter returns data indicating that the GPU should handle the entire composition task .
Consequently, SurfaceFlinger invokes the GPU to perform the composition and sends the result to hwcomposer for submission .
Finally, hwcomposer transmits the buffer via the Wayland protocol to the Linux compositor (mutter), which then sends it to the Linux DPU for display .

               ┌──────────────────────────┐
               │    App (Self-rendering)  │<──────────┐
               └─────┬──────────────┬─────┘           │
                     │              │                 │
Submit Rendering     │              │ Request Buffer  │ Allocate
Results              │              │                 │
                     ▼              ▼                 │
┌─────────────┐ Ask  ┌──────────────┴──────┐     ┌─────┴──────────┐
│ hwcomposer  │<────┤    surfaceflinger    ├────>│    gralloc     │
│ (wayland    │     └──────────────▲───────┘     └────────────────┘
│   client)   ├────────────────────┘
└──────┬──────┘ Respond: ALL DONE BY GPU
       │
       │ Transmit Buffer via Wayland
       ▼
┌─────────────┐
│   mutter    │
│(Composition)│
└──────┬──────┘
       │
       ▼
┌─────────────┐
│   Display   │
└─────────────┘

Of course, this approach entails some performance overhead, but it effectively achieves our objective .

3. Summary & What's Next

These are the two core components of how FDE currently operates . Beyond this, we have also worked on integrating Linux applications into Android, upgrading the UI, fine-tuning interactions, and more .

In the next installment, we'll dive into the technical principles behind FDE's Linux application integration .
"""