Adventures learning Python

I've recently been looking at some open source Python projects and thinking I should give it a go again. I last seriously did Python during university, and since then I've basically been on the Ruby train. But reading through some open source code, I was pleasantly surprised by how readable Python is - enough that I thought I'd try and build a little project in it.

It's been quite a fun journey: Python is similar enough to my most used languages that it doesn't take a huge amount of effort to pick up, but different enough that the differences really stand out and take some thought and research to work around.

The good

Python has types now!

This was quite an exciting find, and made picking the language back up again a lot easier than I expected. It was especially nice to have auto-complete, auto-import and explanatory inline documentation for most of the libraries I added (yes, not all of those require a type system, but they're enhanced by having one).

The usability of the type system feels somewhere between TypeScript (awesome) and Ruby (underwhelming). It's nice to have built-in syntax for parameter and return types - doing this in Python:

def fn(x: str) -> str:
  ...

is much nicer than the equivalents in Ruby type checkers:

# Sorbet, struggling to coerce a type system into Ruby syntax
sig {params(x: String).returns(String)}
def fn(x)
  ...
end

# or RBS, putting the signature in a separate file
def fn: (x: String) -> String

...

def fn(x)
  ...
end

However, there are still some places where type checking feels tacked-on rather than a core feature of the language. It's weird to me that types like str and int can be used without imports, but the Any type requires from typing import Any. The idea of having an opt-in typing package like that is nice, but TypeScript seems to have a more sensible distinction between built-in and imported types.

Named parameters

I was ambivalent on named parameters last time I used Python, but my concern for readability has increased a lot since then and they're now one of my favourite features. JavaScript's explicit object syntax is awkward, Ruby's implementation with separate positional and named parameters better, Swift's internal and external names are fun but clunky, but Python's "any parameter can be used with a name" is a wonderfully simple and flexible implementation.

def listen(port: int = 3000, host: str = '127.0.0.1'):
  ...

# can be used like
listen(host='0.0.0.0') # easily override any parameter
listen(4000) # if it's easy to read, omit the name
listen(port=4000) # if it seems a bit unclear, add the name
# and many more combinations

Where in other languages you need to carefully think "when this is used, will the parameters be easy to understand?" while writing a function, in Python you just give the parameters sensible names and let whoever is using the function decide what will read best.

The 50/50

Imports

I remember when I used to like Ruby's require system where everything gets dumped into the global namespace. After working on some huge Rails projects, I'm now much more of a fan of JavaScript/TypeScript's import system which neatly isolates everything with none of Ruby's name clashes or weird namespacing errors. So I was happy to find that Python's imports work much the same way.

That said, I still don't really understand the import system. I especially don't understand why other projects I've seen can use from ..some.relative.looking.folder import ... and I can't. But I can use from some.absolute.path.in.my.project import ..., which seems more convenient anyway, so I don't know why some projects seem to avoid using that. I also find it a bit weird that imports use dots but effectively refer to folders and files.

So I guess I'm ambivalent. I like the isolation of Python's module system compared to Ruby, but it doesn't feel as immediately understandable as ES Modules - as a relative beginner it takes more background knowledge and context to work out how to create a models/__init__.py and then do from models import stuff compared to just creating models.js and do import stuff from './models.js'. But I have to assume Python's import rules are something you learn once and then aren't a big deal in practice.

The bad

Lambdas are awkward

I'm very used to doing nice functional data manipulation in JavaScript and Ruby, and it's sad to see that Python still can't do that level of expressive data manipulation. Where in Ruby I can do:

users_by_email = responses.map(&:user).group_by(&:email)

in Python I need to write out the lambdas with somewhat verbose syntax, import one of the functions, and write it "backwards" because these are functions rather than methods:

users_by_email = groupby(
  lambda user: user.email,
  map(lambda response: response.user, responses)
)

You can see how this would get complex with longer chains of methods. Obviously you can write this in different ways, but to me it makes the most sense to think in terms of small pieces of logic and visualise the data gradually changing as it flows through a pipeline, and expressing that is frustratingly awkward in Python.

No safe navigation

In web development there's often a lot of interaction with third party APIs where the response structure is ambiguous or changeable, and as far as I've found data access in Python is relatively awkward.

Unlike JavaScript's safe navigation syntax (response?.data?.user?.id) or Ruby's dig method (response.dig(:data, :user, :id)), Python's default dict access feels very limiting. You seem to end up doing a bunch of things like this:

data = response["data"] if "data" in response else None

# or

try:
    id = response["data"]["user"]["id"]
except KeyError:
    id = None

both of which feel long and complex compared to JavaScript and Ruby.

It's a similar story with objects: if you have an object which may or may not be present, there's no easy way to

Libraries like glom seem to alleviate the problem a bit for dicts and arrays, but the general lack of built-in null checking (or None checking, I suppose) feels worrying to me coming from the safety of TypeScript.

Procedural or OO?

One of my favourite things about Ruby is how consistent it is - "everything is an object" is a powerful concept which makes doing pretty much anything in Ruby feel nice and consistent with everything else.

Python doesn't have this, so for someone like me who isn't used to the language it feels confusing why some things are methods on objects and others are just functions. Why is list.count() a method but filter() is a function?

Overall

If you count things up, it seems I have more gripes about Python than things I enjoy. That doesn't tell the full story though; there are plenty of things I didn't mention because they're good enough to just work and stay in the background, which at the end of the day is how things should be in a language.

So although it's still early days, I have a more positive view of Python and I'm curious enough to keep looking for solutions or workarounds for some of those annoyances.