Posted on May 1, 2019

Python 4 - New Function Syntax, Maybe?

#python #syntax

Introduction

Hey! I'm xMikee, a small-time developer who writes Python modules for convenience. Recently, I've been studying the Python syntax and comparing it to the syntax of other languages.

I have been using JavaScript a lot recently to work on my web-based projects. I switch back and forth between which language I use (obviously for different use cases), and it really helps me understand the differences between the two languages. Some notable ones:

Braces - Python relies heavily on indentation
super (no function calling in JS but a function in Python)
this (refers to the object something belongs to in JS)
get/set (special syntax in JS but decorators in Python)
Spread syntax (basically the equivalent of *variable in Python)
Dictionaries (no quotes required for keys in JS, the opposite for Python)

And probably many more I am forgetting. Now obviously, these languages do not share the same principles and therefore the syntax will be very different (JS is C-style, but Python is not). But I'd like to add some aspects of JavaScript into Python. Whaaaaaat? Don't freak out too much, I'll explain.

Let me just start my saying that there is absolutely nothing wrong with Python functions the way they are now. I'm not trying to go out there and propose that we add braces to the language in any way. It would be perfectly fine if this proposal was rejected. I'd like to try and tackle readability issues with functions as parameters. Don't quite get what I'm saying? Here's an example.

Let's say that you are using the Fetch API in JavaScript to do a web request. Here's what your sample code may look like:

fetch('http://example.com/movies.json')
  .then(function(response) {
    return response.json();
  })
  .then(function(myJson) {
    console.log(JSON.stringify(myJson));
  });

Now, let's translate this program to pseudo-Python (obviously fetch is not a Python function, but we're just translating):

import json

def handler1(response):
  return response.json()

def handler2(myJson):
  print(json.dumps(myJson))

fetch('http://example.com/movies.json')
  .then(handler1)
  .then(handler2)

Not quite as pretty, is it? Let's point out the differences here.

In JavaScript, we could define our functions inline. This increased readability since we can see the clear connection in what functions are being processed and how it fits into the flow of the program.
In Python, we had to define separate functions for both of the handlers. Since Python does not directly support inline functions (excepting lambda expressions - we'll cover that later), we had to define separate functions to pass in as the handlers. In a much more complex program, this could quickly get out of hand and be unclear what functions are being executed where.

This seems to be against a few of of Python's core values, defined in the Zen of Python:

Simple is better than complex.
...
Readability counts.

Interesting, right? Now, let's come back to lambda expressions. Lambdas are callable expressions in Python - not functions. Lambdas are good for passing in one-expression methods (for instance, our example above probably could have been converted to two lambdas). But that's pretty much all they're good for.

Once you start requiring more than one expression in your functions, or statements, you'll need to switch over to functions. This is because lambdas are expressions, and therefore only accept expression syntax, not statements. The fact that you can only use one complete expression severely limits the capabilities of lambda expressions in larger-scale programming with more complex operations.

Therefore, we need to use functions in most cases. As we saw above, it's ugly to define them separately just so they can be used as a parameter. There's really no solution to this issue - at least, not yet.

My Proposal

I apologize in advance. The syntax highlighting does display some dark red/black text because the proposed features are technically invalid syntax. Try to ignore the apparent syntax errors.

Due to strict indentation requirements in Python, it would be very difficult to implement a true system of inline functions while still being somewhat attractive and practical. I believe I have found a happy medium between all these options to create an attractive, useful syntax. Here's the example from above, except using the new syntax:

import json

fetch('http://example.com/movies.json')
  .then((def (response):
    return response.json()
  ))
  .then((def (myJson):
    print(json.dumps(myJson))
  ))

Let's pull apart these lines.

.then((def (response):
  return response.json()
))

The entire function is parenthesized. This is required if passing it directly as a function to allow for other arguments to be passed along side the inline function.
The def statement has no name following it - the arguments start immediately.
Indentation is still required and mandatory (and actually fits quite nicely in the arguments, if I must say so myself).
An ending parenthesis ends the inline function. More arguments can be added at this point to the function.

Not too shabby, right? This is what my proposal is. It's called Inline Function Syntax. Distinct from lambda expressions, inline functions are completely capable of performing any tasks a regular function may require.

Here's another example for your viewing. It uses the map function. Previously, in order to write more complex operations, you would have to use a whole separate function instead of a lambda, which is unwieldy. But now, here's the function inlinified:

result = map(range(60), (def (number):
  if number % 2 == 0:
    return "even"
  elif number % 2 == 1:
    return "odd"
  else:
    return "whoa, freaky number"
))

You see? It even looks somewhat attractive! We were able to pass in range(60) as our first parameter, and we could even pass in more arguments after the function if we wanted to, like this:

result = map(range(60), (def (number):
  if number % 2 == 0:
    return "even"
  elif number % 2 == 1:
    return "odd"
  else:
    return "whoa, freaky number"
), another_value)

Of course, inline functions are objects. That means they can be assigned to variables. Here's another example using an inline function, but this time assigning it to a variable:

freaky = def (number):
  if number % 2 == 0:
    return "even"
  elif number % 2 == 1:
    return "odd"
  else:
    return "whoa, freaky number"

result = map(range(60), freaky)

So that's just basically an alternate syntax for function creation. Since inline functions are really just functions in a nutshell, it's not really surprising that they support type hinting/function annotations:

from typing import Iterator

fib = def (n: int) -> Iterator[int]:
  a, b = 0, 1
  while a < n:
    yield a
    a, b = b, a+b

result = fib(60)

Type hints are also supported when passing an inline function as a function parameter due to the fact that indentation is preserved.

There's one more special gotcha, and this is inspired directly from JavaScript (even more than the rest of the syntaxes in this post). It's called Immediately Invoked Function Expressions. That's a direct JavaScript feature (read about in on MDN. You can use it to produce privately scoped variables to perform a calculation. without storing the function afterwards.

Here's an example. Say we want to generate the first 60 Fibonacci numbers, but we want all the variables to be private and we don't want the function stored afterwards. Here's an example:

result = (def (n):
  a, b = 0, 1
  while a < n:
    yield a
    a, b = b, a+b
)(60)

It's important to note that the function must be parenthesized, so that the function and the call can be separated. This can be useful in some instances.

If we're speaking honestly here, I don't think this would ever be implemented, not even a little bit. But it's just another thing I'm proposing to spice up the language. :)

Of course, this would also come with it's own host of incompatibility issues as well. But that's what you get with any interesting syntax change.

Let's hear your thoughts down below. What do you think about functions? Would the Python community benefit from a change like this, or would it hurt the language? Thanks for reading, and here's a cat because you read all the way to the end. 😺

Top comments (8)

Charis Marangos • May 2 '19 • Edited

Hello,

I'm working on a language with Pythonic syntax and I've deliberated about the exact thing: How to deal with inline functions. Turns out it's not as easy as it seems. The reason is how the lexer works, which happens before the grammar is parsed.

Note: I'm going to be collectively referring to parentheses / braces / brackets as just "brace op".

What Python does is something like this:

Skip all whitespace except newline characters
If there's an opening brace op, push it to a stack (each type of brace op has its own stack, so 3 stacks).
If there's a closing brace op, pop from its equivalent stack

The following two happen ONLY IF there is not currently an unclosed brace op:

First non-whitespace character of every line checks for indentation. If the current indentation is different, then yield INDENT or multiple DEDENT tokens accordingly (indentation levels also need to be tracked in their own stack)
Newline characters on lines with non-whitespace characters, and are not preceded by a backslash (\), yield a NEWLINE token.

i.e. Python does not yield NEWLINE or INDENT/DEDENT tokens when there are unclosed brace ops. This makes the implementation of both the lexer and parser much simpler than it would have been otherwise.

You can see that inlining functions would make things more difficult and possibly even create grammatical ambiguities. I decided to opt against a solution for it in my own language, but I might change my mind.

But, here's a possible solution:

Simply, having a new type of brace op will fix this. Maybe something like <: :>. (Visually I prefer <{ }> but that can be grammatically ambiguous). When one of these bad boys is encountered, we get a new set of brace op stacks. i.e. Instead of having 3 stacks, we have a stack where each element of the stack is itself 3 stacks corresponding to parentheses / braces / brackets.

For example, the following code will be much easier to parse than the version without the new brace op:

result = map(range(60), <: def (number):
  if number % 2 == 0:
    return "even"
  elif number % 2 == 1:
    return "odd"
  else:
    return "whoa, freaky number"
:>, another_value)

The problem with any solution is that it makes everything non-pythonic. I'm not sure it's worth implementing.

miniscruff • May 2 '19

Hey Mike, I enjoy these sort of thought experiments and exercises on what a language could look like. As much as I love using anonymous functions in javascript I never really felt like they were very optimal. And often times I find myself getting lost in my own code with the lack of coherent names. I think that pythons style of having named functions is, although maybe a bit jumpy at times, more readable.

I would like to compare your example with a current python version.

result = map(range(60), (def (number):
  if number % 2 == 0:
    return "even"
  elif number % 2 == 1:
    return "odd"
  else:
    return "whoa, freaky number"
))

and my own...

def number_type(number):
    if number % 2 == 0:
        return 'even'
    if number % 2 == 1:
        return 'odd'
    return 'woah, freaky number'


result = map(number_type, range(60))

In your example of using an anonymous function, you must read at least a portion of it to understand what is going on. Whereas in my example, we have given it a name. It may not be the best name but it is a description nonetheless. Now to understand what result is I can simply read the one line and realize that it is a map of the number types from 0-60.

I also find your first example of python a little bit strange. Given that a similar API to fetch in python could be considered requests I think it would look a little closer to this:

import requests
req = requests.get('http://example.com/movies.json')
print(req.json)

If this intent of having inline functions was to allow for more async style callbacks then simply using async and await would do the trick as well.

Finally to your last point, I believe, all python variables in functions are already scoped. And if you wanted to create a function and delete it afterward you can literally del it.

def fib(n):
  a, b = 0, 1
  while a < n:
    yield a
    a, b = b, a+b
result = fib(60)
print(a, b) # a and b are undefined
del fib # fib is now undefined

You could probably wrap this in a decorator or a context manager for single use function, but I am not sure what benefit it will have.

Anyway, sorry if this was too critical it was, of course, a thought experiment.

a • May 2 '19

Hey miniscruff,

I understand that for some, the syntax may not be as practical and intuitive. That's why I believe it will never be implemented at all :)

diek • May 2 '19

Wtf python 4 on the title, you mocked me.

Julian Camilleri • May 4 '19

Oh my f... No. You didn't just compare js syntax with Python 🤦‍♂️ what you don't get that the js syntax should change 😂 if Python changes into that, then why not use Java? ....... Hence why Python is winning over devs. (: Sorry but don't agree with one bit from this article.