DEV Community

Cover image for Parallel File Transfers in Rust — How I Made Android Sync Actually Fast
hiyoyo
hiyoyo

Posted on

Parallel File Transfers in Rust — How I Made Android Sync Actually Fast

#tauri #rust #programming #webdev

All tests run on an 8-year-old MacBook Air. All results from shipping 7 Mac apps as a solo developer. No sponsored opinion.

Sequential file transfer is slow. HiyokoAutoSync uses parallel transfers with a concurrency limit. Here's how I built it.

The problem with sequential transfer

Copy 100 photos from Android to Mac one at a time: each transfer waits for the previous to complete. ADB overhead per file adds up. On a large library, this takes minutes.

Parallel transfers use the available bandwidth more efficiently. Same 100 files, 6 concurrent transfers: significantly faster.

tokio::sync::Semaphore for concurrency control

Unlimited parallelism isn't better — it overwhelms the ADB connection and the device. A semaphore limits concurrent transfers to a useful number:

use tokio::sync::Semaphore;
use std::sync::Arc;

const MAX_CONCURRENT: usize = 6;

async fn transfer_files(files: Vec<FileEntry>) -> Result<(), AppError> {
    let semaphore = Arc::new(Semaphore::new(MAX_CONCURRENT));
    let mut handles = vec![];

    for file in files {
        let sem = Arc::clone(&semaphore);
        let handle = tokio::spawn(async move {
            let _permit = sem.acquire().await.unwrap();
            transfer_single_file(&file).await
        });
        handles.push(handle);
    }

    // Collect results
    for handle in handles {
        handle.await??;
    }

    Ok(())
}
Enter fullscreen mode Exit fullscreen mode

6 concurrent transfers was the sweet spot in my testing — fast without overwhelming the connection.

Progress tracking across parallel transfers

Each transfer needs to report progress independently. Use an atomic counter:

use std::sync::atomic::{AtomicUsize, Ordering};

let completed = Arc::new(AtomicUsize::new(0));
let total = files.len();

for file in files {
    let completed = Arc::clone(&completed);
    let handle_clone = app_handle.clone();

    tokio::spawn(async move {
        let _permit = sem.acquire().await.unwrap();
        transfer_single_file(&file).await?;

        let done = completed.fetch_add(1, Ordering::Relaxed) + 1;
        handle_clone.emit("transfer-progress", Progress {
            completed: done,
            total,
            current_file: file.name.clone(),
        }).ok();

        Ok::<(), AppError>(())
    });
}
Enter fullscreen mode Exit fullscreen mode

Error handling in parallel

One failed transfer shouldn't kill all others. Collect errors and report at the end:

let results: Vec<Result<(), AppError>> = futures::future::join_all(handles)
    .await
    .into_iter()
    .map(|r| r.unwrap_or_else(|e| Err(AppError::Task(e.to_string()))))
    .collect();

let errors: Vec<_> = results.into_iter().filter_map(|r| r.err()).collect();
if !errors.is_empty() {
    // Report partial failure — some files transferred, some didn't
}
Enter fullscreen mode Exit fullscreen mode

Partial success is better than all-or-nothing for file transfers.

The result

6-lane parallel transfer on HiyokoAutoSync is noticeably faster than sequential for large photo libraries. The semaphore pattern is reusable for any parallel work with a concurrency limit.

TL;DR: Use tokio::sync::Semaphore with MAX_CONCURRENT = 6 to parallelize ADB file transfers without overwhelming the connection. Track progress with AtomicUsize, and use join_all with per-error collection so one failed transfer doesn't abort the rest.

If this was useful, a ❤️ helps more than you'd think — thanks!

HiyokoAutoSync | X → @hiyoyok

Top comments (0)