DEV Community

Cover image for Rust's Option type... in Python

Rust's Option type... in Python

TaiKedz on October 13, 2023

Cover Image (C) Tai Kedzierski How many times have you written/seen code like this: data = get_data() print(data.strip()) Enter fullscre...
Read full post
Collapse
 
xtofl profile image
xtofl •

My penny?

Mypy has caught many forgotten None-checks in my code. The Pythonic way seems to be to not bother at runtime, but type-hint, and check statically:

from typing import Optional
def maybe_fruit(i: int) -> Optional[str]:
  return "5" if i == 5 else None

len(maybe_fruit(5))
Enter fullscreen mode Exit fullscreen mode

This will fail to type check:

$ mypy m.py 
m.py:6: error: Argument 1 to "len" has incompatible
   type "str | None"; expected "Sized"  [arg-type]
Found 1 error in 1 file (checked 1 source file)
Enter fullscreen mode Exit fullscreen mode

Note: it does seem that mypy is not fool proof, though.

Another, run-time checked, approach is to implement the Maybe monad. Some articles exist, and some libraries do. E.g. skeptric.com/python-maybe-monad/.

Collapse
 
taikedz profile image
TaiKedz •

Static analysis is one of the ways to try and ferret things out ; it probably does rely on it identifying correctly that a function can in fact return None, which I presume is not fully trivial...

The technique I explored is more "forceful"...

It's an interesting Mybe-monad article you link ; it also links through to a Wikipedia section which... essentially describes the Rust Option itself....

en.wikipedia.org/wiki/Monad_(funct...

Collapse
 
griels profile image
Ellis Breen • • Edited

It still requires the end user to handle the Null special case, rather than the monadic approach where the monad handles it:

def maybe_fmap(f: Callable[a, b]) -> Callable[Optional[a], Optional[b]]:
  return lambda(x): f(x) if x is not None else None
Enter fullscreen mode Exit fullscreen mode

This Option solution just moves the problem - the end user can just forget to check for Nullity and then:

def unwrap(self, message=_NULL_MESSAGE):
    if self.value is None:
        raise Panic(message)``
Enter fullscreen mode Exit fullscreen mode

Is this any better than the AttributeError or a TypeError that'd be raised if attempting to reference/operate on None?

This just increases the boilerplate a user has to use in order to remind them to check null, while doing nothing to force them to. The maybe monad approach saves both typing and enforces correctness, and is composable to boot.

Thread Thread
 
taikedz profile image
TaiKedz • • Edited

Yes - the manifestation of the problem is moved as you point out ;

The point of my musing is not the attempt of making a fool-proof type-safety , but more of a "how's this for signposting?"

Let's say the dev is a user of my library, and I've done the wrapping and this moved-problem type, etc.

What was

data = thelib.some_call()
print(data.strip() )
Enter fullscreen mode Exit fullscreen mode

necessarily becomes

data = thelib.some_call().unwrap() # and by virtue of typing this, alarm bells should have started ringing
print(data.strip() )
Enter fullscreen mode Exit fullscreen mode

The point is, forcing the unwrapping makes the consuming dev explicitly think of handling null, instead of forgetting.

Instead of

# TICKET-123456: fix the None-crash
data = thelib.some_call()
if data is None:
  print("fail")
else:
  print(data.strip() )
Enter fullscreen mode Exit fullscreen mode

it becomes

#data = thelib.some_call().unwrap^[^[^[^[^[^[^[^[^[luckily I got that before shipping
data = thelib.some_call()
if data.is_none():
    print("fail")
else:
    print(data.unwrap().strip())
Enter fullscreen mode Exit fullscreen mode

It's a kludge, of course, don't get me wrong.

And no, I haven't put it in my projects after all ;-)

Thread Thread
 
griels profile image
Ellis Breen • • Edited

I mean, maybe it's better than nothing, though I feel like it gives you a false sense of security. It's akin to a security-by-obfuscation measure. To be fair, annoying the developer in order to make foolish things hard is sometimes necessary as an API or language programmer. I've just been rather frustrated that many of the 'Optional' types (such as Java's Option) still offer a get (equivalent to your unwrap) at the end of the day, allowing you to entirely bypass the purpose of the endeavour, much like one could with the Option class you describe.

Glad you haven't put it in your projects after all.. But it's a nice exercise in trying to replicate Rust's exacting, beautiful compiler and standard library. Obviously Rust and its wonderful ADT support does the heavy lifting in enforcing correctness here.

Collapse
 
michalmazurek profile image
Michal Mazurek •

I like the or_default()

But you could do:

value = some_func() or "default value"
Enter fullscreen mode Exit fullscreen mode

The other option I think would be more pythonic, by just raising an exception instead of returning None.

def some_func(...):
      # ... something produces value
      if value is None:
            raise DoesNotExists("Some meaningful message")
      return value
Enter fullscreen mode Exit fullscreen mode

This whole Result handling in rust comes from lack of exceptions, python do have them, so why not use them?

Collapse
 
taikedz profile image
TaiKedz • • Edited

Indeed, this is completely unpythonic, but for a reason:

In most languages (python being one of a plethora), it is an extremely common mistake to just assume we have a value, until it blows up at runtime with a rogue None.

Hence the "billion dollar mistake," where so many developers forget to handle the None case.

By making it a type, and gating it inside an Option, the idea is to enforce "you cannot do anything with this unless you fully acknowledge that you could have nullity and you have explicitly chosen how to take action"

So instead of null-ness being maybe-or-maybe-not a possibility:

# oops, might be None, dunno, I thought we guaranteed something?
# whatevs, should be fine, didn't see it ...
data = get_data()
Enter fullscreen mode Exit fullscreen mode

the function explicitly says "hey, handle the case, or else!"

# I explicitly acknowledge that I am not handling None
#       because FML YOLO
data = get_data().unwrap()
Enter fullscreen mode Exit fullscreen mode

So really, yes, there are idiomatic pythonic wasy to handle None. Adding the type enforces "You cannot mistakenly fail to handle it"

Collapse
 
michalmazurek profile image
Michal Mazurek •

Yeah, I agree I saw many cases where None was not handled properly. But if we require a specific thing to be returned only from a function, then we as well can ask for handling of None or writing a function in a way that None will never be returned.

On the other hand what floats your boat, if it works it's good!