Python Charming with Toolz

Toolz is an extremely useful collection of utility functions that can help you do things that you do all the time, but more explicitly, and often with less code.

I'm going to highlight a few of the functions that I use the most, but there are many others, and they're all solid gold.

Create a new dict from an existing dict with some items excluded

Given an original dict:

original = {
    'A': 1,
    'B': 2,
    'C': 3,
}

Filtering items by key

To create a new dict from original while excluding the item with key='C', without toolz, I'd probably do:

new = {
    k: v
    for k, v in original.items()
    if k != 'C'
}

Using dicttoolz.keyfilter, this becomes:

new = keyfilter(lambda k: k != 'C', original)

What I really love about this keyfilter solution, aside from being more concise, functional, etc., is that its name clearly tells you what's going on.

Filtering items by value

Given a helper function called is_even that returns a bool indicating whether the single numeric argument is even:

is_even = lambda x: x % 2 == 0

Or, as not a lambda:

def is_even(x):
    return x % 2 == 0

To create a new dict from original while including only the even values, without toolz, I'd probably do:

new = {
    k: v
    for k, v in original.items()
    if is_even(v)
}

Using dicttoolz.valfilter, this becomes:

new = valfilter(is_even, original)

For a single key, get the value from each dict in a list of dicts

Given a list of dicts representing users:

users = [
    { 'name': 'User A',
      'age': 30,
    }, 
    { 'name': 'User B',
      'age': 32,
    }, 
    { 'name': 'User C',
      'age': 41,
    }, 
    { 'name': 'User D',
      'age': 43,
    }, 
]

To get just the ages, without toolz, I'd do:

ages = [ user['age'] for user in users ]

Using itertoolz.pluck, this becomes:

ages = pluck('age', users)

Note that this ^ (like some other toolz functions) actually returns a generator-type itertools.imap object, which is fine if you're going to iterate over it, but if you want to index into it like ages[0], you'll need to convert it to a list (or tuple):

>>> ages[0]
TypeError: 'itertools.imap' object has no attribute '__getitem__'

>>> ages = list(pluck('age', users))
>>> ages[0]
30

You can also pluck multiple keys at once by specifying them as a list:

age_name_pairs = pluck(['age', 'name'], users)

Which looks like:

>>> list(age_name_pairs)
[(30, 'User A'), (32, 'User B'), (41, 'User C'), (43, 'User D')]

Group a list of dicts by some criteria

Given the same list of user dicts as before:

users = [
    { 'name': 'User A',
      'age': 30,
    }, 
    { 'name': 'User B',
      'age': 32,
    }, 
    { 'name': 'User C',
      'age': 41,
    }, 
    { 'name': 'User D',
      'age': 43,
    }, 
]

If I wanted to group the users into decade-resolution age groups (i.e. 30s, 40s, etc.), without toolz maybe I'd do something like:

age_decade_users_map = defaultdict(list)
for user in users:
    age_decade = int(user['age'] / 10) * 10
    age_decade_users_map[age_decade].append(user)

This produces:

>>> dict(age_decade_users_map)
{30: [{'age': 30, 'name': 'User A'}, {'age': 32, 'name': 'User B'}],
 40: [{'age': 41, 'name': 'User C'}, {'age': 43, 'name': 'User D'}]}

Using itertoolz.groupby, this can be accomplished with:

age_decade_users_map = groupby(
    lambda user: int(user['age'] / 10) * 10,
    users
)

Honorable mention goes to:

• itertoolz.first - get the first element from an iterable (even sets!)

• itertoolz.partitional_all - split an iterable into tuples of a specified max length

• dicttoolz.assoc - get a copy of a dict with a specified item added

• dicttoolz.dissoc - get a copy of a dict with specified keys removed (similar in function to my keyfilter example but a better way to do it)

• dicttoolz.get_in - like dict.get() but with support for multi-depth keys, e.g. get_in(['a', 'b'], {'a': {'b': 1}}) = 1

Happy toolzing!