DEV Community

viky
viky

Posted on

1

Concurrent Futures in Python: Launching Parallel Tasks with Ease

Achieving optimal performance through parallel execution is essential. Python, a versatile programming language, provides several tools for concurrent execution. One of the most powerful and user-friendly modules is concurrent.futures, which allows developers to run calls asynchronously. In this article, we'll explore the functionality of this module and how to leverage it for various tasks, including file operations and web requests.

Overview of Concurrent Futures

The concurrent.futures module offers an abstract class known as Executor, which facilitates the execution of calls asynchronously. Although it should not be used directly, developers can utilize its concrete subclasses, such as ThreadPoolExecutor and ProcessPoolExecutor, to perform tasks concurrently.

Key Features

  1. Submit Method: The submit method is where the magic happens. It schedules a callable function to be executed asynchronously and returns a Future object. The callable is executed with provided arguments, allowing developers to run background tasks seamlessly.
   with ThreadPoolExecutor(max_workers=1) as executor:
       future = executor.submit(pow, 323, 1235)
       print(future.result())
Enter fullscreen mode Exit fullscreen mode

In this example, we use a ThreadPoolExecutor to raise a number to a power in a separate thread.

  1. Map Method: The map method is another fantastic feature that allows executing a function across multiple input iterables concurrently. It collects the iterables immediately and executes the calls asynchronously.
   results = executor.map(load_url, URLS, timeout=2)
Enter fullscreen mode Exit fullscreen mode

This functionality is particularly useful when you have a list of tasks that you want to run in parallel.

Practical Application: Copying Files

Consider a scenario where you need to copy multiple files efficiently. The following code snippet demonstrates how to use a ThreadPoolExecutor to copy files concurrently:

import concurrent.futures
import shutil

files_to_copy = [
    ('src2.txt', 'dest2.txt'),
    ('src3.txt', 'dest3.txt'),
    ('src4.txt', 'dest4.txt'),
]

with concurrent.futures.ThreadPoolExecutor() as executor:
    futures = [executor.submit(shutil.copy, src, dst) for src, dst in files_to_copy]
    for future in concurrent.futures.as_completed(futures):
        print(future.result())
Enter fullscreen mode Exit fullscreen mode

This example leverages the shutil.copy function to perform file copies in parallel, significantly improving performance for large-scale file operations.

Handling Web Requests Concurrently

Another exciting application of the concurrent.futures module is retrieving content from multiple URLs at once. Below is a simple implementation using ThreadPoolExecutor to fetch web pages:

import concurrent.futures
import urllib.request

URLS = [
    'http://www.foxnews.com/',
    'http://www.cnn.com/',
    'http://europe.wsj.com/',
    'http://www.bbc.co.uk/',
    'http://nonexistant-subdomain.python.org/',
]

def load_url(url, timeout):
    with urllib.request.urlopen(url, timeout=timeout) as conn:
        return conn.read()

with concurrent.futures.ThreadPoolExecutor() as executor:
    results = executor.map(load_url, URLS, timeout=2)
    for result in results:
        print(result)
Enter fullscreen mode Exit fullscreen mode

This code is a straightforward way to retrieve web content quickly, demonstrating just how easy it is to implement concurrent execution in your projects.

Conclusion

The concurrent.futures module provides a powerful way to execute tasks asynchronously in Python, simplifying the process of achieving parallelism in your applications. Through its Executor class and methods like submit and map, developers can efficiently manage background tasks, whether they involve file operations, web requests, or any other I/O-bound processes.

By incorporating these techniques into your programming practices, you'll be able to create more responsive and efficient applications, enhancing both performance and user experience. Happy coding!

Image of AssemblyAI

Automatic Speech Recognition with AssemblyAI

Experience near-human accuracy, low-latency performance, and advanced Speech AI capabilities with AssemblyAI's Speech-to-Text API. Sign up today and get $50 in API credit. No credit card required.

Try the API

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay