loading...

Discussion on: Why I don't believe in pure functional programming anymore

Collapse
ssokolow profile image
Stephan Sokolow

As someone who came to Rust because he burned out multiple times trying to achieve the kind of confidence Rust's type system brings using Python unit tests, I think I'll have to disagree with that.

A type system may not be a magic wand, but it can rule out classes of bugs and, if it's powerful enough, you can do some really clever stuff.

(eg. Using Rust's type system to verify correct traversal of a finite state machine, as with things like Hyper's protection against PHP's "Can't set header. The body has already begun streaming" runtime error. Rust's ownership is key to that because it ensures you can't accidentally hold and use a reference to an old state.)

Also,

“Program testing can be used to show the presence of bugs, but never to show their absence!”
― Edsger W. Dijkstra

Thread Thread
johncip profile image
jmc

I'm at a disadvantage because I haven't written in Rust, but I would also guess that it's a strict improvement over Python. Python doesn't have good closures, or ways of being point-free, or metaprogramming, etc. So, after giving up type checking, you're not getting back. In even Ruby you'd have to give up a lot more (or perhaps admit a lot more complexity, like Scala's) to add type checking everywhere. (That said, I'm interested in trying Sorbet at some point, which lets you do gradual type checking in Ruby.)

Anyway, I tend to view bugs along a couple of axes. One of them is domain / non-domain. On the domain side, you have bugs that arise from shifting requirements, or misunderstandings of the problem's edge cases. Often these are the most costly to fix, and typically they're harder to discover. Certainly we can't just throw a tool or paradigm at them. On the other end of that spectrum, you have things that happen across codebases -- typos, dereferencing null, etc. These are amenable to tools and disciplines -- type checks, statelessness, immutability, reuse, etc.

I also picture another axis, for just the "low-level" bugs, from static (e.g. typos) to dynamic (e.g. I didn't realize this state was reachable).

Pure FP prevents bugs along the whole axis, but all code gets harder to write, and the more abstract / generic it is, the harder it gets. (In Haskell, mostly it's just hard, in Elm you're actually prevented from writing something like a generic map function, AFAIK.)

Everything else address just one area. Impure FP addresses the state / mutability side. Type-checked imperative code addresses the other. I'd also argue that design-by-contract is somewhere in the middle.

For me, the question becomes, ignoring pure FP, what's more important to prevent? Which constructs are too useful to sacrifice? I tend to prefer having more genericity, easier metaprogramming, etc., so I mostly prefer dynamic + FP. I used to do a lot of Java and don't miss it. (But maybe I should try Rust.)

I also haven't done much with Typescript, core.typed, Sorbet, etc. If I was in a position to add type checking gradually to code I was already writing, I'd probably use that at least some of the time.

But yeah, I feel like the story with Python is that there's a lot of "we have classes, BUT" / "we have lambdas BUT" etc. No type checking but not much of anything else either, in terms of well-executed language constructs. But it's easy to get started with, and that often has a lot of value.

Thread Thread
yujiri8 profile image
Ryan Westlund Author

That's an interesting categorization. But seeing you say this...

I'm at a disadvantage because I haven't written in Rust, but I would also guess that it's a strict improvement over Python.

... makes me want to mention a few things in Python's defense. (I do think Rust is a better language than Python, but far from strictly better.)

I think Python offers the following main benefits over Rust:

  • Much cleaner syntax (I know some people consider significant indentation a bad thing and this can get heated, but I favor it strongly). Comprehensions are as readable as a sentence, and you don't need a whole bunch of "glue" function calls like .to_string() to cast string literals from &str to String.

  • Inheritance (Rust doesn't have it, not even in the form of struct embedding)

  • A REPL. This is huge.

  • Error handling is better in some ways (though I realize it's worse in other ways): you get full stack traces with line numbers for each frame out of the box. No need for third-party crates and writing context messages yourself (except for displaying errors to end users).

  • Default argument values and keyword arguments.

  • A great stdlib and ecosystem. Rust has almost no stdlib and an ecosystem full of immature/abandoned/badly documented creates drowning out the few good ones.

Also, you say Python doesn't have metaprogramming. Are you defining metaprogramming to strictly mean macros? Because it's true Python doesn't have macros, but a lot of their use cases are filled by excellent reflection capabilities. Have you seen libraries like Pydantic (or even the stdlib dataclasses)? They use reflection to accomplish some pretty amazing stuff, like automatic model validation and cutting out all the constructor boilerplate.

I've seen more broad definitions of metaprogramming used before, by which Python has way better metaprogramming than any language without full-on macros.

Thread Thread
johncip profile image
jmc

A REPL. This is huge.

That's a great point, I was forgetting about the lack of REPL in Rust and what a game-changer they are.

Also sad to hear that the Rust ecosystem isn't as good. I think I'd rather have a REPL and good libraries than type-checking, tbh.

Also, you say Python doesn't have metaprogramming.... Python has way better metaprogramming than any language without full-on macros.

Sorry, that was ambiguous, I meant that it doesn't have "good" metaprorgramming, which is perhaps unfair. Certainly Python has metaprogramming. But I think of Ruby's as the benchmark -- send / define_method / missing_method / etc. are very lightweight, and can be used many contexts where something like metaclasses are overkill.

FWIW, I think this is often the story with Python, especially when comparing it with Ruby on a given feature. Generally the Python version will be clever, but narrow, and likely necessary because of a weakness somewhere else.

List comprehensions are a good example -- elegant syntax for the easy 20% of cases, which you pay for in the 80% where map would have been simpler.

By contrast, Ruby's solution is having a lightweight syntax for passing code into functions ("blocks"). Any function can be called with a block -- it's not opt-in, or even opt-out, it's just part of what it means to be a function. So map, filter, and reduce with a closure is very readable, and the standard lib stuff concerned with enumerables is all effortlessly higher-order.

Not only is that great when you build lists, it's great everywhere you use higher-order functions. As another specific example -- you don't need things like Python's context managers, which are sort of heavyweight. Regular ruby functions can already do that.

Also, re: Rust, lack of default arguments (and apparently overloading too) sounds really annoying. Python's evaluation of those at function-define-time is a really, really stupid gotcha, but at least Python has them. Even C has vararg functions. Good lord.

Thread Thread
yujiri8 profile image
Ryan Westlund Author

List comprehensions are a good example -- elegant syntax for the easy 20% of cases, which you pay for in the 80% where map would have been simpler.

Eh? If you ask me, list comprehensions are more readable than map in almost every case. But Python does have map in its prelude - as well as filter, and reduce is in the functools module. (I think there's also some currying stuff in there, which I've never needed.)

Also, comprehensions have serious performance benefits, which I attribute to them not constructing a lambda object and not iterating twice when you do both map/filter in one.

By contrast, Ruby's solution is having a lightweight syntax for passing code into functions ("blocks"). Any function can be called with a block -- it's not opt-in, or even opt-out, it's just part of what it means to be a function. So map, filter, and reduce with a closure is very readable, and the standard lib stuff concerned with enumerables is all effortlessly higher-order.

I've been digging into Crystal lately, which is basically statically typed Ruby. I'm finding the block solution a bit awkward and I'm not sure if it's even as powerful as just allowing functions to be passed to each other. My understanding is that you can't directly pass a function as an argment in Crystal or Ruby because just referencing its name calls it with 0 arguments. Crystal and Ruby have multiple different kinds of these things - blocks, procs, stabby lambdas - where Python just has functions and lambdas. The block syntax, while nice, seems restricted in use case to me because (unless I'm mistaken) it only allows passing a single block.

FWIW though I do see that Python's lambda story is a bit weak because they can only return something, and inline defs (which are prohibited at least in Crystal) don't handle scoping in the most desirable way. My ideal story in this department is Javascript (though I kind of hate to admit it, since I hate everything else about Javascript).

Not only is that great when you build lists, it's great everywhere you use higher-order functions. As another specific example -- you don't need things like Python's context managers, which are sort of heavyweight. Regular ruby functions can already do that.

Yeah, I realized yesterday that blocks are the equivalent of with, but I'm not sure if I like them more. I don't think context managers are heavyweight though. The contextlib.contextmanager decorator can turn a function into one; the function does its setup in try and yields the resource, and its cleanup in finally.

Also, re: Rust, lack of default arguments (and apparently overloading too) sounds really annoying. Python's evaluation of those at function-define-time is a really, really stupid gotcha, but at least Python has them. Even C has vararg functions. Good lord.

Lol. I agree that the evaluation of defaults at compile time is a stupid gotcha, but as for vararg functions, I'm not actually sold on them. What's the benefit compared to functions that take arrays?

Thread Thread
johncip profile image
jmc

Python does have map et al, but lacks the anonymous inner functions which would make them more useful for building & reducing collections.

A few things:

  • Blocks aren't merely the equal of with -- they're more like lambdas where return acts on the outer function. So (like lambdas) they cover the with case, the collection-wrangling cases, and many others.
  • You can pass multiple blocks -- proc makes blocks, lambdas, and even named functions first-class. But the "default block" has more convenient syntax.
  • I don't think it's fair to say that Python has one way to do it (lambdas) while other languages have multiple. I'd say Python has zero, unfortunately. Python's lambdas are anonymous inner expressions while every other language uses the term to mean anonymous inner functions. Likewise, some variants of BASIC have "user-defined functions" which are limited to single statement. Beats not having them, but still not as useful as real functions.

[I edited the above to group them and clarify what I mean by lambdas]

One more example -- there's no "reduce" comprehension in Python. So you're back to loops there. They give you a sum function, but you don't have the thing that would let you write your own sum function elegantly.

I should probably stop... it's hard to talk about these things convincingly because often the problem with Python isn't with what it has, but the fact that it needed to have those things in the first place. Along those lines, yes the generator-style context managers are lighter-weight than classes, but languages with anonymous functions don't need to provide "generator-style context managers" in the first place. Anyway, I don't want to come off as hating Python. It's just behind the times in some ways.

as for vararg functions, I'm not actually sold on them. What's the benefit compared to functions that take arrays?

Not much, since the varargs is syntactic sugar. But it lets you be explicit about intent -- sometimes you really are passing in single argument that happens to be an array. Varargs let you say "this isn't an array, but a bunch of discrete arguments that I will figure out what to do with at runtime."

Thread Thread
yujiri8 profile image
Ryan Westlund Author

One more example -- there's no "reduce" comprehension in Python. So you're back to loops there. For the specific case of adding, they give you a sum function. But you don't have the thing that would let you write your own (at least in a functional style).

Huh? I just mentioned functools.reduce. It works with any function or lambda. What does Ruby's reduce get that Python's doesn't?

Thread Thread
yujiri8 profile image
Ryan Westlund Author

Oh - is it the ability to reduce from the outer function (halting iteration)?

I guess that could be useful... in some really obscure situation or something...

Thread Thread
johncip profile image
jmc

Oh I just meant Python's comprehensions -- they cover some of the places you'd use map & filter, but there's no comprehension for going down to a scalar, AFAIK. (That said, since reduce can be used to build collections, list / dict / set comprehensions do end up overlapping with it.)

And to be clear, when I said "you're back to loops" I meant for the places where you wouldn't bother to create a named inner function, or to compose named functions before calling them. (It's not that you don't have options, but I'd argue that they're not idiomatic or lightweight.)

What does Ruby's reduce get that Python's doesn't?

While reduce works the same everywhere, it's most useful in languages with anonymous inner functions. It's true of higher-order functions in general, not just reduce.

Ruby goes a step further by providing a shorthand syntax for doing this (but is not unique there -- Clojure has the #() shorthand, and JS has ->).

FWIW, since you mentioned it -- I've only felt the need to halt from inside a reducer function in Clojure. I forget why, but it probably had to do with starting from an infinite stream. It'd be an optimization to prevent the creation of intermediate collections.