The latest articles on DEV Community by Desmond Cheong (@desmondcheongzx). https://dev.to/desmondcheongzx https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3574426%2F41801085-6a5f-4653-b555-672255b9a41b.png https://dev.to/desmondcheongzx en Desmond Cheong Sun, 19 Oct 2025 22:08:55 +0000 https://dev.to/desmondcheongzx/benchmarking-multimodal-ai-workloads-daft-vs-spark-vs-ray-data-3i41 https://dev.to/desmondcheongzx/benchmarking-multimodal-ai-workloads-daft-vs-spark-vs-ray-data-3i41 <p>Multimodal AI workloads (audio, video, images, documents) break traditional data engines. The team at <a href="https://www.daft.ai/" rel="noopener noreferrer">Daft</a> released a <a href="https://www.daft.ai/blog/benchmarks-for-multimodal-ai-workloads" rel="noopener noreferrer">benchmark of their engine against Spark and Ray Data across four real-world multimodal workloads</a>:</p> <p>Audio transcription (113K files with Whisper)<br> Document embedding (10K PDFs)<br> Image classification (803K ImageNet images with ResNet18)<br> Video object detection (1K videos with YOLO11n)</p> <p>Results: Daft ran 2-7x faster than Ray Data and 4-18x faster than Spark, completing jobs reliably without failures. On the heaviest workload (video detection), Spark took over 3.5 hours while Daft finished in under 12 minutes.</p> <p>The benchmark code and logs are fully open-sourced for reproducibility. If you're building multimodal AI pipelines, this is worth checking out.</p> dataengineering deeplearning performance ai