DEV Community

Tugay PALA
Tugay PALA

Posted on

I rendered FreeRDP through Metal to make remote Windows feel native on macOS

#macos #swift #rdp #ssh

The problem: remote Windows on a Mac feels like a slideshow

I write networking software for a living, and my daily setup is a few Linux servers over SSH plus one Windows box I have to RDP into. On macOS the official option is Microsoft's Remote Desktop app (now "Windows App"), and it works — but under load it often feels like watching a slideshow. Scrolling a long log, dragging a window, anything with motion, and the latency is right there in your face.

I wanted to know why, and whether I could do better inside my own app. This post is the honest version of what I learned.

First attempts: CoreGraphics and IOSurface

My first RDP prototype decoded frames from FreeRDP and pushed them to the screen as CGImages. It worked on paper and was miserable in practice: every frame meant a CPU-side copy and a full layer redraw. On a 1440p remote session the main thread was the bottleneck long before the network was.

I moved to IOSurface to cut some copies out. Better, but I was still fighting the compositor for every update, and partial-update regions (the thing that makes RDP efficient) weren't buying me much because I kept re-uploading whole frames.

What actually worked: CAMetalLayer + a Metal texture

The version that finally felt right treats the remote desktop as what it is — a texture that changes over time.

  • A CAMetalLayer backs the view.
  • FreeRDP decodes into a buffer; I upload it into an MTLTexture.
  • Damaged regions from the RDP graphics pipeline become partial texture updates instead of full re-uploads.
  • A trivial fragment shader draws the texture to a full-screen quad.

Suddenly the CPU is mostly idle, the GPU does the compositing it's built for, and remote Windows scrolls the way it should. Scrolling long logs and dragging windows at 1440p went from visibly choppy to fluid — the difference is obvious the moment you grab a window and shake it.

The lesson wasn't "Metal is magic." It was: stop treating a video-like stream as a pile of images, and let the part of the machine designed for textures do the work.

The SSH side had its own surprises

For SSH I used Apple's swift-nio-ssh. Clean, modern, pure-Swift — and missing keyboard-interactive auth, which a lot of real-world servers still lean on. My workaround so far: when there's no saved password (or it's rejected), I pause the handshake and prompt the user for a password right there, then retry password auth — which covers most servers that would otherwise trip on it. True keyboard-interactive support is still blocked at the library level. If you're building on NIO SSH, budget time for the auth edge cases; the happy path is lovely, the long tail is where the work is.

Why put SSH, SFTP and RDP in one window

Once the RDP engine was smooth, keeping it in a separate app made no sense. My real workflow is: watch a deploy on three servers, drop a file onto one over SFTP, jump to the Windows box to check something. Three apps, three Cmd-Tabs.

So I built the whole thing into one native SwiftUI app — sessions laid out in a grid so I can see several at once, dual-pane SFTP with drag-and-drop, and RDP in the same window. I called it Noden.

There's also an optional AI assistant, but with a deliberate constraint: it proposes a command and waits for me to approve it. It never runs anything on its own. Bring your own API key or use the built-in one, and turn it off entirely if you don't want it.

Honest limits, because this audience deserves them: it's macOS only, there's no VNC yet, and no iOS app. If you need cross-platform, other tools fit better.

Try it / tell me I'm wrong

If you've shipped an RDP or SSH client and solved these differently, I genuinely want to hear it — the Metal path was trial and error and I'm sure there's more to squeeze.

Top comments (0)