DEV Community

Dave Banwell
Dave Banwell

Posted on

Python has map() and filter()? How come no one told me?

As a new python developer, fresh off my first major project, I'm pretty excited about the language. So excited that I can't resist learning more and more by watching content from seasoned developers. Recently, I came across a video titled "map() and filter() in Python!" This piqued my interest because I hadn't been taught these inbuilt functions; I had only ever been taught how to approximate these using list comprehensions.

I thought to myself "Surely an inbuilt function for such a thing MUST be better than typing out an entire comprehension!" After watching the video and learning the functions, I still felt unclear on whether or not these functions would be superior to list comprehensions. So I decided to put them to the test, head to head.

Product

Right off the bat, it's important to not that the inbuilt map() and filter() functions return a generator object, whereas a list comprehension returns a list. The functions were starting off on a great foot since generator objects are way more memory efficient than lists. But, generator objects come with their share of downsides. If you need a snapshot of your collection as it looks based on the parameters at the time you generated it, lists are the way to go since they were created at the moment the list comprehension was assigned to its variable. But generator objects don't actually "create" their constituent values at the time of assignment, but rather when they get iterated over. So it's possible that the inputs to the list comprehension could change, giving you unexpected values upon iteration. At the end of the day, as long as you stay mindful of this, you could probably use whichever one fit your needs and you'd be fine.

Keystroke

I had always considered list comprehensions to be rather cumbersome to type. I thought, perhaps, that I might have to type less when using the inbuilt functions. Wow, was I wrong! As it turned out, list comprehensions required fewer keystrokes than using map() or filter(). Take a look at a simple map() operation compared to its comprehension counterpart:

sum(map(lambda x: x * x, TEST_NUMS))
Enter fullscreen mode Exit fullscreen mode 
sum([x * x for x in TEST_NUMS])
Enter fullscreen mode Exit fullscreen mode

Now consider that lambda functions can only handle simple operations, while list comprehensions can go further, including filtering right in your map operation. When it comes to keystrokes, list comprehensions are king. Look at the below for comparison:

TEST_NUMS = range(10_000_000)

def square_even_nums(num):
    if num % 2 == 0:
        return num ** 2
    else:
        return 0

def map_it():
    return sum(map(square_even_nums, TEST_NUMS))
Enter fullscreen mode Exit fullscreen mode 
TEST_NUMS = range(10_000_000)

def comp_it():
    return sum([x * x for x in TEST_NUMS if x % 2 == 0])
Enter fullscreen mode Exit fullscreen mode

In the above examples, using the map() function required 10 lines and 193 characters, whereas using the list comprehension required 4 lines and 102 characters. If you're concerned with verbosity, stick to list comprehensions.

Efficiency

This is where things get really interesting because generator objects are certainly more memory-efficient than lists. On top of that, consider cases where you're grabbing the first value in the collection. Using the next() function to obtain that from a generator object is considerably faster (at nanosecond scale) than using the pop() function on a list, or even obtaining the value at this_list[0].

In reading other posts and articles about the differences between these functions and list comprehensions, it seemed to be the consensus that list comprehensions were faster in cases where lambda functions were used to support map() and filter(), but that map() and filter() generally coe out on top where other inbuilt functions are used. I wanted to test this for myself and to try it on an array of different scenarios.

In terms of simple operations, where lambda functions can be used, comprehensions came out on top. In my testing, they were consistently 25-30% faster than their function counterparts.

TEST_NUMS = range(10_000_000)

def map_it():
    return sum(map(lambda x: x * x, TEST_NUMS))

def comp_it():
    return sum([x * x for x in TEST_NUMS])
Enter fullscreen mode Exit fullscreen mode 
Daves-MacBook % python -m timeit -s "from map_or_comp import map_it" "map_it()"  
1 loop, best of 5: 632 msec per loop

Daves-MacBook % python -m timeit -s "from map_or_comp import comp_it" "comp_it()"
1 loop, best of 5: 496 msec per loop
Enter fullscreen mode Exit fullscreen mode

When it came to named functions, I was quite surprised at how well map() and filter() handled other inbuilt functions as callbacks. They performed the operations significantly more quickly than list comprehensions did.

Pythonic-ness... Pythonicity.... Is it Pythonic?

The consensus seems to be that list comprehensions are more pythonic than the inbuilt functions. They're shorter, more aparently explicit, more easily readable and - in most cases - quicker and more efficient.

So... Should you use map() or filter()?

I've seen different advice from different long-time developers. Some advise folks to never use them. Others advise folks to use them only when passing another inbuilt function as a callback. As for me, taking into account that comprehensions were released after the functions, that they are - in most cases - superior in the ways that matter most, and that schools are eschewing even teaching the functions in favour of list comprehensions, I'd say it's about time that we deprecate the functions in favour of the more versatile, more pythonic list comprehensions.

Top comments (0)