DEV Community

Avnish
Avnish

Posted on

How do I split a list into equally-sized chunks?

Splitting a list into equally-sized chunks is a common task in Python programming, especially when dealing with batch processing or dividing data into manageable parts. Let's delve into three different ways to achieve this, including examples with detailed explanations and outputs.

Example 1: Using yield for a Generator Function

This method is efficient for large lists since it generates chunks on the fly without storing the entire list of chunks in memory.

def split(list_a, chunk_size):
    for i in range(0, len(list_a), chunk_size):
        yield list_a[i:i + chunk_size]

# Parameters
chunk_size = 2
my_list = [1,2,3,4,5,6,7,8,9]

# Splitting the list and printing
print(list(split(my_list, chunk_size)))
Enter fullscreen mode Exit fullscreen mode

Explanation:

  • split() is defined to take a list (list_a) and a chunk_size.
  • Inside the function, a for loop iterates over the list using range() with steps of chunk_size, effectively moving chunk_size elements forward in each iteration.
  • yield then returns a chunk from list_a[i:i + chunk_size] where i is the current index. This creates a generator, making this method memory-efficient.
  • Finally, list(split(my_list, chunk_size)) converts the generator into a list to print the chunks.

Output:

[[1, 2], [3, 4], [5, 6], [7, 8], [9]]
Enter fullscreen mode Exit fullscreen mode

Example 2: Using List Comprehension

List comprehension provides a compact way to achieve the same result.

chunk_size = 2
my_list = [1,2,3,4,5,6,7,8,9]
list_chunked = [my_list[i:i + chunk_size] for i in range(0, len(my_list), chunk_size)]
print(list_chunked)
Enter fullscreen mode Exit fullscreen mode

Explanation:

  • Here, [my_list[i:i + chunk_size] for i in range(0, len(my_list), chunk_size)] creates chunks of size chunk_size. It iterates over the list with a step size of chunk_size, slicing the list from i to i + chunk_size for each chunk.
  • This approach directly stores the chunks in list_chunked and prints them.

Output:

[[1, 2], [3, 4], [5, 6], [7, 8], [9]]
Enter fullscreen mode Exit fullscreen mode

Example 3: Using NumPy's array_split

This method is particularly useful when working with numerical data or when you are already using NumPy for data processing.

import numpy as np

my_list = [1,2,3,4,5,6,7,8,9]
# Splitting into 5 chunks
chunks = np.array_split(my_list, 5)
print(chunks)
Enter fullscreen mode Exit fullscreen mode

Explanation:

  • np.array_split(my_list, 5) splits my_list into 5 chunks. Unlike splitting into equal sizes, array_split can handle cases where the list cannot be divided evenly, distributing the extra elements as evenly as possible.
  • The result is a list of NumPy arrays.

Output:

[array([1, 2]), array([3, 4]), array([5, 6]), array([7, 8]), array([9])]
Enter fullscreen mode Exit fullscreen mode

Additional Methods:

Beyond these examples, Python offers other approaches for specific use cases, such as using libraries like more_itertools.chunked or implementing custom functions for specific types of data.

These examples showcase the versatility of Python in handling list operations, offering multiple ways to split lists into chunks according to the requirements of different scenarios.

Top comments (0)