DEV Community

Umesh Adabala
Umesh Adabala

Posted on

Using your Android Device from your Mac

#android #python #productivity #showdev

I built MacDroid, an open-source macOS utility that lets you connect an Android phone to your Mac, mirror its screen in real time, and interact with it using your Mac's keyboard, mouse, and trackpad.

The idea started with a simple question:

What would a clean Android-to-Mac experience look like?

There are already excellent technologies for individual parts of this problem. Android provides ADB, macOS provides powerful desktop APIs, and the open-source ecosystem provides mature tools for video streaming and decoding.

I wanted to bring those pieces together into a focused desktop application.

The result is MacDroid v1.

Repository: https://github.com/umeshadabala/macdroid

What is MacDroid?

MacDroid is a lightweight desktop application designed to make an Android phone usable directly from a Mac.

The basic workflow is intentionally simple:

Connect Android phone
        ↓
Launch MacDroid
        ↓
Detect device
        ↓
Start Android session
        ↓
Mirror screen
        ↓
Control Android from Mac
Enter fullscreen mode Exit fullscreen mode

The first version focuses on the core interaction experience.

You can:

  • Mirror the Android screen in real time
  • Control Android using a mouse
  • Use a Mac trackpad for gestures
  • Type using the Mac keyboard
  • Use Android navigation controls
  • Capture screenshots
  • Detect connected Android devices
  • Connect through USB
  • Monitor connection and performance information
  • Configure display and performance settings

The philosophy behind v1 was straightforward:

Connect the phone. Open MacDroid. Start using it.

Why Build This?

The interesting part of a software project is often the problem behind it.

I kept thinking about how useful it would be to treat an Android phone as another interactive device attached to a Mac.

Not just a screen.

Not just a file-transfer target.

An actual interactive Android environment that can be controlled from the desktop.

That means being able to:

  • See the phone
  • Click on the phone
  • Scroll through applications
  • Type into applications
  • Navigate Android
  • Capture the screen
  • Continue working from the Mac

That immediately creates an interesting systems problem.

Multiple components need to cooperate:

Android Device
      │
      │ ADB
      ▼
Device Session
      │
      ├── Video Stream
      │       │
      │       ▼
      │   H.264 Decoder
      │       │
      │       ▼
      │   Frame Renderer
      │
      └── Input Channel
              │
              ├── Mouse
              ├── Trackpad
              └── Keyboard
Enter fullscreen mode Exit fullscreen mode

The project became an opportunity to explore how these pieces could be combined into a clean desktop application.

The Architecture

MacDroid v1 is structured around several major components.

At a high level:

                    ┌─────────────────────┐
                    │      MacDroid       │
                    │     Qt / PySide6    │
                    └──────────┬──────────┘
                               │
              ┌────────────────┼────────────────┐
              │                │                │
              ▼                ▼                ▼
        Device Layer      Video Layer       Input Layer
              │                │                │
              ▼                ▼                ▼
             ADB          H.264 Stream     Mouse / Trackpad
                              │             / Keyboard
                              ▼
                           Decoder
                              │
                              ▼
                           Renderer
Enter fullscreen mode Exit fullscreen mode

The major responsibilities are separated into:

UI
├── Main Window
├── Mirror View
├── Toolbar
├── Status Information
└── Settings

Device
├── Device Detection
├── ADB Communication
└── Device Session

Video
├── Video Stream
├── H.264 Data
├── Decoder
└── Frame Renderer

Input
├── Mouse Controller
├── Trackpad Controller
├── Keyboard Controller
└── Coordinate Mapper

Utilities
├── Configuration
├── Logging
└── Process Management
Enter fullscreen mode Exit fullscreen mode

This separation is important because video processing, input processing, and UI rendering have very different responsibilities.

Why Python?

For v1, I chose:

Python + PySide6

instead of immediately building the entire application using Swift/AppKit.

There were several reasons for this.

First, Python gives me a very fast development cycle.

I wanted to experiment with:

  • Device communication
  • Input handling
  • Video streaming
  • Coordinate mapping
  • UI structure
  • Performance

without spending most of the initial development time on platform-specific implementation.

Second, Python can work very well as an orchestration layer.

The performance-sensitive parts don't necessarily need to be implemented as pure Python operations.

For example:

Python
   │
   ├── Application logic
   ├── Device management
   ├── Input translation
   └── UI orchestration
            │
            ▼
     Native libraries
            │
            ├── FFmpeg
            └── Video decoding
Enter fullscreen mode Exit fullscreen mode

This allowed me to keep development productive while using native components where appropriate.

The Desktop UI

The UI is built with PySide6 / Qt.

I wanted the application to feel like a small macOS utility rather than a large management dashboard.

The Android display is the primary focus.

The surrounding UI is intentionally compact.

The core toolbar contains actions such as:

Back   Home   Recents   Screenshot
Enter fullscreen mode Exit fullscreen mode

Alongside connection information and performance information.

The design principle was:

The interface should support the phone rather than compete with it.

That means fewer unnecessary controls and more space dedicated to the mirrored device.

Connecting Android

Android provides the Android Debug Bridge, commonly known as ADB.

ADB is extremely useful for development and device-management workflows because it provides a communication layer between the host computer and an Android device.

The basic flow is:

Mac
 │
 │ USB
 ▼
Android Device
 │
 ▼
ADB
 │
 ├── Device Detection
 ├── Commands
 └── Input / Session Communication
Enter fullscreen mode Exit fullscreen mode

MacDroid uses ADB as the foundation for device discovery and communication.

The application can detect a connected Android device and establish a session before starting the mirroring pipeline.

The USB-first approach also gives the first version a straightforward and predictable transport layer.

Screen Mirroring

A screen-mirroring application has two fundamental jobs:

  1. Receive the Android display stream.
  2. Decode and render it on the Mac.

Conceptually:

Android Display
      │
      ▼
Encoded Video
      │
      ▼
Transport
      │
      ▼
MacDroid
      │
      ▼
H.264 Decoder
      │
      ▼
Video Frame
      │
      ▼
Qt Renderer
      │
      ▼
Mac Window
Enter fullscreen mode Exit fullscreen mode

The video stream is encoded using H.264.

H.264 is a practical choice for this type of workload because it provides efficient video compression while having mature decoder implementations available across platforms.

The goal isn't simply to transfer screenshots repeatedly.

A continuous encoded video stream is much more suitable for real-time interaction.

H.264 Decoding

One important design decision was avoiding a workflow where every frame gets converted into a large Python image object and processed manually.

A naive approach could look something like:

Video Frame
    ↓
Python Image
    ↓
NumPy Array
    ↓
Color Conversion
    ↓
Qt Image
    ↓
Display
Enter fullscreen mode Exit fullscreen mode

Doing this repeatedly can introduce unnecessary CPU and memory overhead.

Instead, MacDroid uses PyAV, which provides Python bindings around FFmpeg's multimedia capabilities.

The simplified pipeline becomes:

H.264
  ↓
FFmpeg / libavcodec
  ↓
PyAV
  ↓
Decoded Frame
  ↓
Qt Rendering
Enter fullscreen mode Exit fullscreen mode

This lets the application take advantage of mature native multimedia infrastructure while retaining Python as the application layer.

Input Is Just as Important as Video

A mirrored phone isn't particularly useful if you can only look at it.

The real goal is interaction.

That means MacDroid has to translate desktop input into Android input.

There are three major categories:

Mac Input
   │
   ├── Mouse
   ├── Trackpad
   └── Keyboard
           │
           ▼
     Input Translation
           │
           ▼
      Android Input
Enter fullscreen mode Exit fullscreen mode

Each one has different requirements.

Mouse Control

A mouse click on the Mac needs to become a touch interaction on Android.

That sounds simple:

Mac X,Y → Android X,Y
Enter fullscreen mode Exit fullscreen mode

But the actual relationship is more complicated.

The Mac window may be:

  • Larger than the phone
  • Smaller than the phone
  • A different aspect ratio
  • Resized dynamically
  • Displayed with letterboxing
  • Running on a Retina display

So MacDroid needs to understand the actual video rectangle.

Conceptually:

Mac Window
┌───────────────────────────────┐
│                               │
│      ┌───────────────┐        │
│      │               │        │
│      │ Android       │        │
│      │ Video         │        │
│      │               │        │
│      └───────────────┘        │
│                               │
└───────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

The click position needs to be transformed relative to the actual rendered video area.

A simplified transformation is:

normalized_x = (mouse_x - video_left) / video_width
normalized_y = (mouse_y - video_top) / video_height
Enter fullscreen mode Exit fullscreen mode

Then those normalized coordinates can be mapped to the Android source resolution:

android_x = normalized_x × source_width
android_y = normalized_y × source_height
Enter fullscreen mode Exit fullscreen mode

In practice, the transformation also needs to account for things such as orientation and display scaling.

This makes coordinate mapping one of the important pieces of the input architecture.

Orientation

Phones can change orientation.

For example:

Portrait

┌──────────┐
│          │
│ Android  │
│          │
└──────────┘
Enter fullscreen mode Exit fullscreen mode

can become:

Landscape

┌──────────────────┐
│                  │
│     Android      │
│                  │
└──────────────────┘
Enter fullscreen mode Exit fullscreen mode

The video dimensions and input coordinate system can therefore change.

MacDroid's input mapping works with the current source dimensions rather than assuming a fixed orientation.

This becomes especially important when the user resizes the desktop window at the same time.

Retina and HiDPI

macOS introduces another interesting detail.

The logical size of a Qt widget isn't necessarily identical to the number of physical pixels used by the display.

On a Retina display, scaling can affect coordinate calculations.

So input handling needs to distinguish between:

Logical UI Coordinates
Enter fullscreen mode Exit fullscreen mode

and:

Physical Display Coordinates
Enter fullscreen mode Exit fullscreen mode

This is another reason why simply scaling mouse coordinates based on window width and height isn't sufficient for a polished experience.

Trackpad Gestures

Trackpads introduce another category of interaction.

A Mac trackpad can generate high-resolution scroll events.

Android applications, however, generally expect touch-like interactions.

For example:

Mac Trackpad
     │
     ▼
Scroll Gesture
     │
     ▼
Normalize Gesture
     │
     ▼
Translate Movement
     │
     ▼
Android Interaction
Enter fullscreen mode Exit fullscreen mode

The objective is to preserve the intent of the gesture rather than blindly forward every raw event.

A small trackpad movement should produce a small interaction.

A larger movement should produce a larger interaction.

The translation layer therefore normalizes the gesture before sending it to Android.

Keyboard Input

Keyboard support makes MacDroid much more useful for text-heavy Android applications.

Instead of reaching for the phone whenever a text field appears, you can type using the Mac keyboard.

The input flow becomes:

Mac Keyboard
      │
      ▼
Qt Key Event
      │
      ▼
Keyboard Mapper
      │
      ▼
Android Key Event
Enter fullscreen mode Exit fullscreen mode

Common keys such as:

  • Enter
  • Backspace
  • Delete
  • Tab
  • Escape
  • Space
  • Arrow keys

need appropriate handling.

Modifier combinations also require translation between desktop and Android input models.

This is an area where having a dedicated input layer pays off because keyboard behavior can evolve independently from the rest of the application.

Android Navigation

MacDroid also provides direct Android navigation controls.

The basic actions include:

Back
Home
Recents
Enter fullscreen mode Exit fullscreen mode

These are exposed as compact controls in the desktop interface.

The goal is to make common Android navigation actions available without requiring the user to physically interact with the phone.

Screenshots

Screenshots are another useful feature.

When the user captures a screenshot, the goal is to save the Android display as an image at the appropriate device resolution.

The basic workflow is:

Android Frame
     ↓
Capture
     ↓
PNG
     ↓
Local Mac Storage
Enter fullscreen mode Exit fullscreen mode

This makes the feature useful for:

  • Development
  • Documentation
  • Debugging
  • Testing
  • Tutorials
  • App demonstrations

Device Detection

A desktop utility should make the initial connection experience as simple as possible.

Instead of asking the user to manually configure a device every time, MacDroid monitors the ADB device state.

Conceptually:

ADB
 │
 ├── No Device
 │
 ├── Device Connected
 │
 ├── Device Ready
 │
 └── Device Disconnected
Enter fullscreen mode Exit fullscreen mode

The UI can then reflect the current state.

For example:

Connected via USB
Enter fullscreen mode Exit fullscreen mode

or the appropriate disconnected state.

This provides a much clearer user experience than requiring users to understand the underlying ADB process.

Performance

Performance was an important consideration throughout the project.

There are several things happening simultaneously:

Android
   │
   ├── Video
   │
   └── Input
          │
          ▼
       MacDroid
          │
     ┌────┴────┐
     ▼         ▼
  Decode     Input
     │         │
     ▼         ▼
  Render     Android
Enter fullscreen mode Exit fullscreen mode

The video pipeline needs to continuously receive and decode frames.

At the same time, input events need to be handled without being blocked by rendering work.

This is why keeping responsibilities separated is important.

The application shouldn't treat the entire process as one large synchronous loop.

Keeping the UI Responsive

A desktop application's UI should remain responsive while background work is happening.

Operations such as:

  • Device detection
  • ADB communication
  • Video processing
  • Session management

should not unnecessarily block the main Qt event loop.

The architecture therefore treats the UI and underlying device/video operations as separate concerns.

Conceptually:

Qt Event Loop
      │
      ├── UI
      │
      └── Signals / Events
               │
               ▼
       Background Operations
               │
               ├── ADB
               ├── Video
               └── Device Session
Enter fullscreen mode Exit fullscreen mode

This makes the application feel much more like a desktop utility.

Why Not Build Everything From Scratch?

One of the biggest lessons from this project was that building a product doesn't mean reinventing every component.

Android already provides ADB.

FFmpeg already provides mature multimedia infrastructure.

Qt already provides a powerful desktop UI framework.

PyAV provides access to FFmpeg capabilities from Python.

Instead of recreating all of these systems, MacDroid focuses on connecting them.

That gives the architecture a useful principle:

Build the product-specific layer yourself. Reuse mature infrastructure where it makes sense.

Development Workflow

The project evolved through incremental testing.

Rather than attempting to build the complete application in one pass, the core system was developed in stages.

A simplified progression looked like:

1. Detect Android device
        ↓
2. Establish USB communication
        ↓
3. Start screen streaming
        ↓
4. Display video
        ↓
5. Add mouse input
        ↓
6. Add keyboard input
        ↓
7. Add Android navigation
        ↓
8. Improve coordinate mapping
        ↓
9. Add trackpad interaction
        ↓
10. Add screenshots
        ↓
11. Refine UI
        ↓
12. Stabilize v1
Enter fullscreen mode Exit fullscreen mode

This approach made it possible to validate each layer before building the next one.

The Importance of Coordinate Systems

One of the more interesting technical areas in MacDroid was coordinate transformation.

There are effectively multiple coordinate systems involved:

Mac Window Coordinates
        ↓
Qt Widget Coordinates
        ↓
Rendered Video Coordinates
        ↓
Normalized Coordinates
        ↓
Android Source Coordinates
        ↓
Android Touch Coordinates
Enter fullscreen mode Exit fullscreen mode

Each layer can introduce its own transformation.

For example, suppose the Android source is:

1080 × 2400
Enter fullscreen mode Exit fullscreen mode

but the rendered video area is:

405 × 900
Enter fullscreen mode Exit fullscreen mode

A mouse position inside the Mac window cannot simply be multiplied by a constant based on the window size.

The application first needs to determine whether the pointer is inside the actual video area.

Then it can calculate its relative position.

Then it can map that relative position to the Android source.

This is a small example of a general principle in interactive systems:

Always transform coordinates based on the actual rendering context, not assumptions about the container.

Designing for a Small Utility

Another important decision was keeping MacDroid focused.

Desktop applications can easily accumulate controls.

A project starts with:

Mirror
Enter fullscreen mode Exit fullscreen mode

Then becomes:

Mirror
Files
Clipboard
Device Manager
Automation
Logs
Settings
Accounts
Cloud Sync
Wireless
Multi-device
Enter fullscreen mode Exit fullscreen mode

Eventually, the original purpose becomes harder to find.

For v1, I wanted the primary experience to remain:

Android Screen
        +
Keyboard
        +
Mouse
        +
Trackpad
Enter fullscreen mode Exit fullscreen mode

Everything around that should support the experience rather than dominate it.

What I Learned

Building MacDroid reinforced several things I enjoy about software engineering.

1. Small ideas can become interesting systems problems

The initial idea was simple:

Mirror Android on a Mac.

But implementing that properly touches:

  • Networking and device communication
  • Multimedia
  • Video codecs
  • GUI development
  • Input systems
  • Coordinate transformations
  • Operating system behavior
  • Performance engineering

A small product idea can therefore become a surprisingly deep engineering exercise.

2. Interaction matters as much as rendering

Getting a screen onto a desktop is only the beginning.

The experience becomes much more useful when the user can actually interact with it.

That means:

Video
+
Input
+
Latency
+
Correct coordinate mapping
+
Good UI
=
Usable product
Enter fullscreen mode Exit fullscreen mode

The interaction layer deserves the same attention as the rendering layer.

3. Existing infrastructure is powerful

A lot of development time can be saved by building on mature technologies.

Instead of implementing:

  • A custom Android communication protocol
  • A custom video codec
  • A custom desktop UI framework
  • A custom multimedia decoder

MacDroid combines existing infrastructure into a product-specific experience.

That is one of the most useful lessons I took from the project.

4. Architecture matters early

Even for a relatively small application, separating responsibilities makes development easier.

When the input layer is separate from the video layer, I can improve input behavior without restructuring the rendering pipeline.

When device management is separate from the UI, the interface can evolve without rewriting ADB handling.

Good boundaries make experimentation much easier.

MacDroid v1

After bringing all of these components together, MacDroid reached its first complete version.

The v1 experience now includes:

Android Device
      │
      ▼
     ADB
      │
      ▼
 MacDroid Session
      │
 ┌────┴───────────────┐
 │                    │
 ▼                    ▼
Video                Input
 │                    │
 ▼              ┌─────┼─────┐
H.264            │     │     │
 │              Mouse Trackpad Keyboard
 ▼
Decoder
 │
 ▼
Qt Renderer
 │
 ▼
Mac
Enter fullscreen mode Exit fullscreen mode

The result is a compact desktop application that brings Android interaction directly into the Mac environment.

What's Next?

MacDroid v1 establishes the foundation.

There are many directions I want to explore from here.

Some of the areas I'm interested in include:

  • Wireless connectivity
  • More advanced macOS integration
  • Multi-device support
  • Improved device compatibility
  • Additional Android-to-Mac interactions
  • More automation capabilities
  • Further performance improvements
  • Native macOS packaging and distribution
  • More polished device/session management

The goal isn't to add features simply for the sake of adding features.

The goal is to continue improving the experience around the central idea:

Make Android and macOS work better together.

Open Source

MacDroid is open source and available on GitHub:

https://github.com/umeshadabala/macdroid

The repository contains the source code, setup instructions, architecture, and development information.

If you're interested in:

  • Android development
  • macOS development
  • Python
  • PySide6
  • ADB
  • FFmpeg
  • H.264
  • Desktop applications
  • Device communication
  • Input systems
  • Open-source software

I'd love for you to explore the project.

Issues, ideas, feedback, and contributions are welcome.

Final Thoughts

MacDroid started as a simple question:

What would it take to make an Android phone feel like a natural part of a Mac workflow?

That question led to experiments with ADB, H.264 video streaming, FFmpeg, PyAV, PySide6, input translation, coordinate systems, trackpad gestures, keyboard handling, and desktop application architecture.

Eventually, all of those individual pieces became one application.

That is what makes building software exciting to me.

An idea starts as a few words.

Then it becomes a prototype.

Then a collection of experiments.

Then a system.

And eventually, you can open it and actually use it.

MacDroid v1 is now shipped.

The project is available here:

https://github.com/umeshadabala/macdroid

And this is only the beginning.

Top comments (0)