DEV Community: Dave Banwell

Python has map() and filter()? How come no one told me?

Dave Banwell — Wed, 21 May 2025 02:28:19 +0000

As a new python developer, fresh off my first major project, I'm pretty excited about the language. So excited that I can't resist learning more and more by watching content from seasoned developers. Recently, I came across a video titled "map() and filter() in Python!" This piqued my interest because I hadn't been taught these inbuilt functions; I had only ever been taught how to approximate these using list comprehensions.

I thought to myself "Surely an inbuilt function for such a thing MUST be better than typing out an entire comprehension!" After watching the video and learning the functions, I still felt unclear on whether or not these functions would be superior to list comprehensions. So I decided to put them to the test, head to head.

Product

Right off the bat, it's important to not that the inbuilt map() and filter() functions return a generator object, whereas a list comprehension returns a list. The functions were starting off on a great foot since generator objects are way more memory efficient than lists. But, generator objects come with their share of downsides. If you need a snapshot of your collection as it looks based on the parameters at the time you generated it, lists are the way to go since they were created at the moment the list comprehension was assigned to its variable. But generator objects don't actually "create" their constituent values at the time of assignment, but rather when they get iterated over. So it's possible that the inputs to the list comprehension could change, giving you unexpected values upon iteration. At the end of the day, as long as you stay mindful of this, you could probably use whichever one fit your needs and you'd be fine.

Keystroke

I had always considered list comprehensions to be rather cumbersome to type. I thought, perhaps, that I might have to type less when using the inbuilt functions. Wow, was I wrong! As it turned out, list comprehensions required fewer keystrokes than using map() or filter(). Take a look at a simple map() operation compared to its comprehension counterpart:

sum(map(lambda x: x * x, TEST_NUMS))

sum([x * x for x in TEST_NUMS])

Now consider that lambda functions can only handle simple operations, while list comprehensions can go further, including filtering right in your map operation. When it comes to keystrokes, list comprehensions are king. Look at the below for comparison:

TEST_NUMS = range(10_000_000)

def square_even_nums(num):
    if num % 2 == 0:
        return num ** 2
    else:
        return 0

def map_it():
    return sum(map(square_even_nums, TEST_NUMS))

TEST_NUMS = range(10_000_000)

def comp_it():
    return sum([x * x for x in TEST_NUMS if x % 2 == 0])

In the above examples, using the map() function required 10 lines and 193 characters, whereas using the list comprehension required 4 lines and 102 characters. If you're concerned with verbosity, stick to list comprehensions.

Efficiency

This is where things get really interesting because generator objects are certainly more memory-efficient than lists. On top of that, consider cases where you're grabbing the first value in the collection. Using the next() function to obtain that from a generator object is considerably faster (at nanosecond scale) than using the pop() function on a list, or even obtaining the value at this_list[0].

In reading other posts and articles about the differences between these functions and list comprehensions, it seemed to be the consensus that list comprehensions were faster in cases where lambda functions were used to support map() and filter(), but that map() and filter() generally coe out on top where other inbuilt functions are used. I wanted to test this for myself and to try it on an array of different scenarios.

In terms of simple operations, where lambda functions can be used, comprehensions came out on top. In my testing, they were consistently 25-30% faster than their function counterparts.

TEST_NUMS = range(10_000_000)

def map_it():
    return sum(map(lambda x: x * x, TEST_NUMS))

def comp_it():
    return sum([x * x for x in TEST_NUMS])

Daves-MacBook % python -m timeit -s "from map_or_comp import map_it" "map_it()"  
1 loop, best of 5: 632 msec per loop

Daves-MacBook % python -m timeit -s "from map_or_comp import comp_it" "comp_it()"
1 loop, best of 5: 496 msec per loop

When it came to named functions, I was quite surprised at how well map() and filter() handled other inbuilt functions as callbacks. They performed the operations significantly more quickly than list comprehensions did.

Pythonic-ness... Pythonicity.... Is it Pythonic?

The consensus seems to be that list comprehensions are more pythonic than the inbuilt functions. They're shorter, more aparently explicit, more easily readable and - in most cases - quicker and more efficient.

So... Should you use map() or filter()?

I've seen different advice from different long-time developers. Some advise folks to never use them. Others advise folks to use them only when passing another inbuilt function as a callback. As for me, taking into account that comprehensions were released after the functions, that they are - in most cases - superior in the ways that matter most, and that schools are eschewing even teaching the functions in favour of list comprehensions, I'd say it's about time that we deprecate the functions in favour of the more versatile, more pythonic list comprehensions.

I'm hooked!

Dave Banwell — Tue, 05 Nov 2024 06:37:19 +0000

And if you’ve struggled with poor state management, I suggest that you get hooked too.

I’ve been flirting with coding for as long as I’ve been able to read. It’s always brought me a lot of joy and – even as a data guru/amateur coder – it’s even proven to be a differentiator for me, in the workplace. But, as an amateur coder, I have felt some frustrations based on my limited knowledge, and – frankly – the limits of the tools I was working with (I’m looking at you, VBA).

I can’t count the number of projects I’ve built where I would have given my last dollar for a simple, straightforward, out-of-the-box way to manage state with little overhead. But, instead, I plodded along with a ton of variables, different setter expressions for every situation, and code that was basically unreadable by anyone who wasn’t me. What a nightmare!

It's not what happens to you, but how you React, that counts

When I got my hands on React, I had no idea what I was in for. From the outset, it made web-app development much simpler and cleaner than the HTML/CSS/JavaScript paradigm. But when I learned about hooks, I realized that my prayers had been answered and that someone, somewhere understood and appreciated my frustrations.

Hooks, for folks who’ve never worked with React, are powerful tools that ‘hook into’ React’s lifecycle and state management features from within a functional component. In essence, a hook is a function that holds data in state and returns data and/or other useful items. All React hooks names are rendered in camel case and begin with the word ‘use’. This convention makes hooks instantly recognizable.

React has some simple and brilliant built-in hooks:

useState – the most fundamental of all – enables a developer to store a piece of data in state and returns variables pointing to the data, itself, and a setter function to change the value of the data in state. This can act as the control center for something as mundane as a dark mode/light mode toggle variable all the way to an object containing the values of all data inputted by a user into a controlled form. This simple hook forms the bedrock that underpins every other hook.
useEffect enables developers to run code, in the form of a callback function, as a side effect of the component’s lifecycle events (mount and unmount) and as a side effect of changes to the values of state variables. Let’s say a username is stored in state; useEffect could be employed to run whenever the username changes, to say good-bye to the previous user and to greet the new one.

A hook isn’t just a catch—it’s a lifeline - ChatGPT

As I began to use these simple hooks, and a few others, I began to realize just how easy they made my life, as a developer. And, as I progressed in my course, I learned something that has changed the way I code in every other language that I use: custom hooks. Yes, with various combinations of useState, other React hooks, and some ingenuity, a developer can write their own hooks for any stateful operation.

I had been having anxiety about a controlled form that updated state on the change in value of any of 2 dozen fields, causing a re-render on every. single. keystroke. I had been having nightmares about a simple button that allowed users to decrement a value by one, causing a fetch request to the server on every. single. click. I thought that there must be a way to throttle the onChange listeners in the form or the onClick listener on the button, delaying re-renders and/or fetch operations until the user was finished inputting. That’s when I learned about useDebounce, a custom hook that tracks multiple clicks or keystrokes and only sends it to the server or sets it to state when the clicking or typing stopped for at least 2 seconds. Hallelujah!

When I found out how relatively simple it was to create something like this, I began to look at other custom hooks that other folks had written and there are some brilliant and useful ones out there. And, finally, I tried my hand at writing my own custom hook – useFetch.

The more that I’ve played around with these fantastic little gems, the more my thinking about state management changed. I didn’t realize it until I was at work several days later, as I approached a problem in VBA, for an Access project. I realized that I needed a hook. I also realized that, if I created a variable in the global scope, set its default on execution start, and defined a setter function for that variable… I basically had a useState hook (albeit in 3 pieces). And, because my React training hit home the importance of never updating state variables directly, naming them and their setter functions, accordingly, keeping the declaration at the highest level in the scope… good state management practices were baked into my methodology.

I’m under no illusions that I’ll ever get VBA to be great with state management. It’s just not built for that. But learning about something as seemingly simple, like hooks, taught me not only how to use them and create them, but also how to think in terms of their utility when designing code in other, less-accommodating languages.

Picking up coding after a long hiatus

Dave Banwell — Wed, 10 Jul 2024 23:13:32 +0000

10 print “David is great!  ”;
20 goto 10
run

That momentous 2-line program and simple command were the first things that I ever typed into a computer, in 1980. My grandmother had borrowed a Commodore PET computer for the summer, from the school where she taught and, over that summer, my aunts taught 5-year-old me the basics of coding in… well… BASIC. They couldn't have known, at the time, that they were laying the foundation for a lifelong obsession with logic, analysis, and problem solving.

In grades 7 and 8, I learned even more about BASIC in a formal setting: variables and variable types, loops, decisions, etc. And, by this time I had my own home computer for practice. I always felt excited when being presented with a new challenge to solve through this cutting-edge technology. In high school, we learned to program simple games, like hangman and craps. It seemed like nothing was impossible with this one, simple language.

In university, I learned HTML and later taught myself CSS (version 2). I loved building simple websites for people and even made some money doing it on the side. But technology started to broaden and advance and – with a full-time job with an insurance company – I didn’t really have the time to upgrade my skillset and I left programming behind.

About 20 years into my career at the same insurance company, I ran into a real problem: I was the sole keeper and maintainer of a large and very important dataset. I was going on vacation for more than a month and there was no one else who could run the weekly updates. My boss tasked me with training someone, but there wasn’t enough time. Knowing the process, I went to Google for a solution and, voila… VBA.

The more that I used VBA in my role as a reporting analyst, the more I grew to depend on it. Not only did it give me a far greater skillset in terms of data mining, but it enabled me to do my job a LOT faster. I’d find myself getting lost in code and loving it. I needed more of this in my life!

And, so, here I am at the end of Phase 1 of Flatiron School’s software engineering bootcamp! I am so completely in love with this program, and I’ve only begun to scratch the surface. With such a long gap in my coding experience, I’m just amazed at a couple of things:

How much has stayed the same
How much has changed

I’m blown away that many of the basic concepts that comprise JavaScript are identical to those underpinning BASIC from 40+ years ago. Learning a brand new language seemed so daunting at first. But, from day 1, it was like riding a bicycle. It was certainly a newer bike with a heaps more features, but it did the same basic things and was incredibly simple to grasp.

My experience with HTML and CSS was somewhat similar. I had a great deal of familiarity with them 20 years ago and relearning them was a snap! But, as much as the fundamentals have remained the same, there are some major and hugely beneficial differences.

When I originally learned web programming, we coded everything in HTML, from the text to the formatting, to the positioning of elements (anyone remember frames?). Dynamic content was virtually unheard of. CSS was emerging, but its main focus was formatting text elements. If you wanted animations, you had a choice between the tag or learning Macromedia Flash. Still, HTML did all the heavy lifting.

I was shocked in the best of ways to see the even, three-way distribution of duties among HTML, CSS, and JavaScript, today. With HTML handling text and text structure, CSS handling all formatting and positioning, and JavaScript working its magic on each of them to produce dynamic content, not to mention interacting with servers and enabling some basic data processing, it seems like anything is possible. And we’ve only just scratched the surface.

My greatest surprise has been just how much functionality is baked into CSS. The sheer volume of properties and values that one can apply to elements and classes is staggering to someone who had only used it for font colours and sizes, previously. I’m thrilled to have so many new tools in my toolbox and I am challenged to use them judiciously.

I remain as excited, today, at running an application of my own making as I had been as a 5-year-old, pressing the key to run that small, but mighty, program at the top of this post. And I know – and relish the thought – that I still have so much to learn.

I have two pieces of advice for anyone who's considering picking up coding again, after a long hiatus:

It is truly never too late.
It's not nearly as daunting as it might seem.

Happy coding!