How do you track your progress?

Ilya Katun — Sat, 16 Jul 2022 20:20:37 +0000

Hi! I fill kinda stuck when I think of what I know and what I don't know (amount of different fields, terms, technologies, etc. in IT is just HUGE).

I want to know, if I'm not the only one with such problem and maybe get some advice or feedback on how you manage that. I'd be grateful if you answer following questions:

Do you have problems with describing/visualizing your knowledge/skills?
Do you need monitor and plan you growth?
If you do something about it, what you do? What instruments do you use?

If you have such problem as a Team Lead with your team I'm also very interested in your feedback!

Curring in Python and why it sucks

Ilya Katun — Tue, 24 May 2022 12:55:08 +0000

What is curring?

Curring is a functional technic (or feature) that allows developers to pass arguments to the functions partially. Let's look at the example.

Example

Imagine we have function decode that converts str to bytes with some encoding.

Yep, I know that there is built-in str.decode method, but for simplicity and some non-imaginary add example let's look at this one

def decode(encoding: str, s: str) -> bytes:
    return s.decode(encoding)

This function is executed like decode("UTF-8", some_str) all the time. I'm sure that you would mostly use one encoding ("UTF-8" in particular) so this "UTF-8" argument would be everywhere. One day tired of writing it all the time you decide to refactor that and write another function:

def decode_utf_8(s: str) -> bytes:
    return decode("UTF-8", s)

It is actually just a shortcut, but unknowingly you used curring (but not in the way real functional languages do)! Curring is about passing some of the arguments to the function to get another function that just waits when you pass left arguments.

So if function has 3 arguments and you pass only 2, than instead of raising exception you actually get function that accepts only one left argument.

What if Python supported curring?

If Python would support this feature natively than you could've written something like:

# this does not work!!!
decode_utf_8 = decode("UTF-8")  # type: Callable[[str], bytes]

Looks great to me, but to be honest I don't see any possibility this might actually become real in Python3.X (maybe in Python4 some day, but who knows?) due to backward compatibility issues.

But there are a few legal ways of curring functions in Python right now and right there.

Option 1: `functools.partial`

With functools.partial I could've written decode_utf_8 like so:

from functools import partial

decode_utf_8 = partial(decode, "UTF-8")

decode_utf_8("hello")  # b"hello"

Option 2: `toolz.curry` (better one)

Another option is provided by great package toolz

from toolz import curry

@curry
def decode(encoding: str, s: str) -> bytes:
    return s.decode(encoding)

decode_utf_8 = decode("UTF-8")

This option looks much-much better as it seems like curring is natively supported with just one decorator.

But why it sucks?

Python is dynamically typed language and after using partial or @curry you can't really get type hint for your code editor. All your attempts to keep code clean and typed via writing type hints and using static type checkers like mypy are lost in vain. VS Code for example tells that decode_utf_8 has type partial or curry instead of actual Callable[[str], bytes].

This makes it hard to use, because you need to remember by yourself what arguments you need to pass to actually execute function. Also it's not always clear what to do with optional arguments. Functions with arbitrary arguments can't be obviously curried too.

Is there any hack?

Well, I've been looking for one and made this simple workaround:

from typing import (
    Callable,
    Concatenate,
    ParamSpec,
    TypeVar,
)

P = ParamSpec("P")
A1 = TypeVar("A1")
AResult = TypeVar("AResult")


def hof1(func: Callable[Concatenate[A1, P], AResult]):
    """Separate first argument from other."""

    def _wrapper(arg_1: A1) -> Callable[P, AResult]:
        def _func(*args: P.args, **kwargs: P.kwargs) -> AResult:
            return func(arg_1, *args, **kwargs)

        return _func

    return _wrapper

# also hof2 and hof3 decorator for separating first 2 and 3
# arguments correspondingly

@hof1
def decode(encoding: str, s: str) -> bytes:
    return s.decode(encoding)

decode_utf_8 = decode("UTF-8")
# but you can't do this:
decode("UTF-8", "hello")  # raises error

With this decorator you must know beforehand what arguments you might want to pass first and what arguments you gonna pass to actually execute function. This is a serious limit, but it helps me save my type hints which I consider more important than easier syntax.

More extensive example:

from toolz import pipe


@hof1
def encode(encoding: str, b: bytes) -> str:
    return b.encode(encoding)


@hof1
def split(sep: str, s: str) -> Iterable[str]:
    return s.split(sep)

@hof1
def cmap(
   mapper: Callable[[X], Y], iterable: Iterable[X]
) -> Iterable[Y]:
    return map(mapper, iterable)

query = pipe(
    b"name=John&age=32",
    encode("UTF-8"),
    split("&"),
    cmap(split("=")),
    dict,
)

print(query)  # {"name": "John", "age": "32"}

# same without hof1
query = pipe(
    b"name=John&age=32",
    lambda b: b.encode("UTF-8"),
    lambda s: s.split("&"),
    lambda ss: map(lambda s: s.split("="), ss),
    dict,
)

What do you think about curring and such workaround? Share your thoughts and questions in the comments!

DEV Community: Ilya Katun