DEV Community

loading...
Cover image for 11 Ways of Applying a Function to Python Pandas DataFrame

11 Ways of Applying a Function to Python Pandas DataFrame

Satish Chandra Gupta
Machine Learning Practitioner. I learn & write about doing ML in production. Cofounder: SlangLabs.in. Ex: Amazon, Microsoft Research. Newsletter: ML4Devs.com
Updated on ・1 min read

Applying a function to rows of a Pandas DataFrame is one of the most common operations during data wrangling. There are many ways of doing it.

I plotted the performance of various ways of applying a function to each row of a Pandas DataFrame, for up to a million rows.

Performance of various alternative for applying a function to Pandas DataFrame

I was surprised to see itertuples() beating apply(), and humble list comprehension beating them both.

So far, I was using apply() whenever I found vectorization difficult. Somehow I thought it was the 2nd best option.

I have been using "%timeit" often. In this exercise, I learned how to do line-level profiling in Python and also plotting the performance over input size.

Continue reading »

Discussion (0)