Why Raku Junctions cannot work - the short story

Or: paying to lay landmines in your own garden.

DISCLAIMER: this article assumes essential knowledge of Junctions in the Raku programming language.

Prelude

Originally, I had 3 topics that I wanted to write about with regards to Raku, to feel that I finished my business with the language.

• Allomorphs. This seemed like a simple but boring topic: they are more a core API design choice than strictly speaking a language feature (it's still not really clear why they were added to core - see my speculations in the article), they are known but not really exciting or hyped, and all I wanted is to show they implement a well-understood design anti-pattern for negligible benefits. Originally, I wanted this to be the second topic of the three because I didn't find it so relevant but since there was already a note and apologetics of Allomorphs in Rakudo Weekly, I thought why not do it right away.
• Junctions - this is today's article. As I mentioned in my previous article, Junctions are a peculiar and popular feature within the Rakuverse. Originally meant as the first topic, I thought it would be good to poke this bubble somewhat because I had already encountered some shortcomings. As I was working on an article around May, it spiralled out of control and I haven't finished it to this day. It was meant to be an exhausting analysis of the topic. This article is only about the problems that, I think, reduce the feature into the most banal use-cases.
• Modularity and dependency management: I'm not sure any article about this topic will happen because it's basically a cluster of topics and I don't want to publish poorly reasoned articles. As an insight: I do think there are certain things that Raku does better than the vast majority of mainstream programming languages, for example the concept of modules ie. smart enough namespaces to actually be published.

TL;DR verdict of Junctions

Junctions are a feature that everybody hears about in Raku, everybody likes at first, and under lucky circumstances, never has to move past the trivial use-cases where they are safe and handy. These trivial use-cases can be summarized as: using exactly one Junction in an expression that is known to auto-thread over the eigenvalues right away, a use-case which could be handled by a compile-time transformation entirely, at a fraction of development and maintenance cost, completely without "runtime support". For anything else, it's either annoying to use Junctions, or confusing to the point of being unsafe. To put it more harshly: imagine that you want pyrotechnics for a New Year's Eve barbecue (why not, sounds kinda nice) and you end up paying 99% of the costs for laying landmines in your own garden. Ouch.

Junction-agnostic code?

First, we should determine what the aim of Junctions as a feature is. The general sentiment is well expressed by this article about them: the premise (promise) is that people can design code without explicitly planning for them and still produce code that is usable very well with Junctions. The most prevalent use-case could then be passing Junctions to code that doesn't know about their existence and supposedly auto-thread over them to produce a new Junction value, much to the caller's delight. Let's walk through the traps!

Rule 1: don't take a Junction by accident

For the sake of code snippets, let me first demonstrate that it's pretty easy to involuntarily accept Junctions in a signature:

• for subroutines, the default parameter type is Any. That's nice but not all values outside Any are Junctions: one quick example would be the constant IterationEnd. When somebody accepts Mu (that is, "truly any") values, it's hard to assume they are actually prepared for Junctions
• worse yet: Blocks, including "pointy blocks" with an explicit signature, default to Mu parameters outright. I'm not sure why that is but this is what I'm going to rely on in my snippets.

The problem with receiving a Junction involuntarily is that it breaks fundamental invariants. To name a few (where $x holds a Junction):

• $x % 2 == 0 and $x % 2 == 1 can be true at the same time - or false at the same time
• after asserting $x < 5, you cannot assume that $x >= 5 would be false
• you cannot assume anything about the number of elements in a Hash/Map like eg. %($x => 42, $y => 53)
• multiple references of $x in an expression would go nuts: $x * $x would produce vastly different numeric results from $x ** 2

These are just examples. As you can see, Junctions work sufficiently different from what you expect from "a value" - this in itself isn't necessarily a bad thing but again, to reiterate the point: you wouldn't want to get them by accident, you want to deliberately plan for them - or avoid them.

I like algebra. Here is a simple lambda to check for the roots of a quadratic polinomial:

my &root = {$_ ** 2 - 3 * $_ + 2 == 0};

The roots are, of course, 1 and 2. Or so I thought... Raku seems to have a mind of its own:

so root(1&2); # False

Hm. I guess that has to be because one (exactly one, even) of the roots is 0 or 2/3:

so root(0^2/3); # True

Since the lambda referred to a Junction value multiple times in the same expression, we got something akin to a Cartesian product of the eigenvalues, hence the false negatives and false positives.

Signatures matter. And since Lists (and Arrays) contain Junctions just fine by default, you better watch out either with your higher-order functions like map or first, or defend against Junctions at data type level.

On the Junction-aware side

Okay, so it's still easy to do the right thing and just pre-emptively not allow Junctions in, right? Raku has never been known as a defensive language, and the main intended use-case seems to be to have Junctions very ephemerally, with almost all code just doing auto-threading over them.

Rule 2: avoid != or unlearn De Morgan's Law

Enter "negation lifting", the thing only ever understood by Larry Wall himself. I feel like beating the dead horse here because there has been plenty of incentive to somehow break free of this and it always seemed to me that Larry Wall was the last and only one who really "got this". In Rakudo, even today, after all these years:

so any(1, 2) ∉ (2, 3); # True - doing auto-threading as you would expect
any(1, 2) !(elem) (2, 3); # False - doing auto-threading over (elem) and then negating and collapsing the Junction (NB no "so")

The supposed equivalents are not equivalent, and if you wanted the latter behavior, you could easily do the negation by hand no problem.

Of course this also makes the previous point more emphatic:

my &safe-remainder = { $_ % 5 != 1 }; # want to disallow 1 modulo 5 values
safe-remainder(any(1, 2, 3, 4, 5)); # False
safe-remainder(all(1, 2, 3, 4, 5)); # True

The behavior wouldn't be any better if we did the comparison in an expression without the involvement of a Block - this is just one peculiarity that you need to learn, even if you are deliberately working with Junctions.

There are just so many problems with "negation lifting": it's linguistically biased and even then, only really stands in the something != any ... case (not even the any .... != something case!), it breaks De Morgan's Law, and the language doesn't always give you a nice way out.

I didn't want to omit this problem but I almost feel petty for covering it: it's the inertia and "design fatigue" that keeps this behavior, I really doubt there is anybody in the community who would defend it.

Rule 3: give up on controlling how Junctions compose - or write boilerplate

So you say, at the end of the day, the problem with "negation lifting" was basically the same as Callables that weren't prepared to deal with Junctions: they didn't follow the semantics of auto-threading.

Anyway, up until now, we avoided talking the semantics of expressions with Junctions altogether. The docs state they are meant to be "boolean matchers". I will go a step further: they are a way to codify first-order logic in Raku, with some additional quantifiers (one, none) built in.

Junctions serve both as the quantifier and a reference to the variable bound by the quantifier. Granted, in "general-purpose code", most of the time, you don't need more than one quantifier (in fact, usually you need none, and therefore no Junctions at all). However, I think 2-quantifier configurations ("for all x there is y such as..." kind of predicates) are common enough to be noted. Here is a code example:

sub valid-ratio($dividend, $divisor) { 1 < $dividend / $divisor < 10 }
so valid-ratio(any(10, 75, 4), all(2, 6, 7, 11)) # True

If I wanted this expression to mean "there is a dividend for which all divisors land us in the right range", I might get a bit confused here: 4 and 10 are too small (when divided by 11) and 75 is way too large (when divided by anything but 11).
Also note sub here: we follow rule 1 and auto-thread over all Junctions.

However, Raku gives you no control over how two different Junctions compose: when you have an operation with any and all, it will always be any-inside-all, ie. a conjunction of disjunctions. So to stick with our example: the code does "for all divisors, there shall be a dividend that lands us in the right range". (And indeed, eg. 10/2, 10/6, 10/7, 75/11.)

This is fairly confusing in an expression even if you happen to have wanted the conjunction-of-disjunctions composition - and if you didn't, you have no choice but to manually create two layers of auto-threading, like this for example:

sub (Any $x, Junction $y) { valid-ratio($x, $y) }(any(10, 75, 4), all(2, 6, 7, 11)).so # False - $x stays the outmost layer

Here, the signature of the IIFE forces that the first argument is collapsed before the second one, therefore moving to the outer layer of composition.

Expressions with Junctions are fairly easy to read as long as you only have one but once you have two, you lose any sort of natural intuition, and potentially control over the first-order predicate you are creating.

Rule 4: avoid multiple references - or write boilerplate

As a continuation of the previous point: we have seen that Junctions can be mapped to first-order logic but they don't give much control over the composition of quantifiers. In fact, they don't really give you much control over the references either: whenever you introduce a Junction in an expression, there is a one-to-one correspondence between the quantifier and the reference to its bound variable.

We have already seen this with $_ * $_ in Rule 1: what you get from any(4, 9) * any(4, 9) == 36 is ∃x∃y: x*y = 36, and you cannot easily tune Junctions to mean ∃x: x*x == 36 - part of the reason is that structures like andthen or .& follow the Block rules and do not collapse Junctions, therefore you carry not just the eigenvalue reference but the whole quantifier+variable nature of the Junction.

So what do you do when you want to check ∃x: x < 10 ∧ x ≡ 1 (mod 5)? I don't know a better way with Junctions than

so sub ($_) { $_ < 10 && $_ % 5 == 1 }(6|7|67); # True

This is definitely not the elegance we were looking for with Junctions, though.

Rule 5: did I mention to avoid multiple references?

This is the one I showed in the previous article.

so 0 < any(-5, 10) < 5; # True

I honestly didn't investigate if this is only a problem with chaining operators - operators where identical associativity breaks down into conjunctive expressions, like 0 < $x < 5 into 0 < $x && $x < 5.

The point is, you may not even see the two quantifiers and references your code is using under the hood. Again, if you previously thought all examples are convoluted and unrealistic: do you think something this simple should be a footgun?

And more importantly, do you still think Junctions are worth the trouble messing up the type system and the signature binding process, having unique composition rules, and both sides of a call paying attention to what they might do?

What's left now

First off, just like I'm still waiting with Allomorphs to come across some good use cases or the original rationale, I would be curious to see good ways to resolve the apparent issues I mentioned, that aren't essentially "either don't use them, or write IIFE boilerplate to manually handle collapsing them".

I also want to mention that without any Junctions ever, often with less code than the escape hatches I found, you could just use already existing race and hyper utilities of the language to enable "threading" (although not "auto") and easily describe the predicates that Junctions apparently sought to emulate closer.

So what's left from the possible uses of Junctions that should "just work without you paying much attention"?

• they should rarely ever cross function barriers; they should rather remain on the caller's side
• even on the caller's side, there shouldn't be more than one, collapsed in an easy-to-reason-about fashion, to be worth it
• the caller should be well aware of the implications of certain expressions regarding the "no more than one" bit

After all this time, I'm tempted to think AlexDaniel has been right all along. Bluntly put, all good uses of Junctions follow the $value (check) $junction or $junction (check) $value pattern. All you would need to cover these cases is a macro, if you are so ashamed of just using a higher-order predicate like Javascript or Python do.

The agenda

I'm not much of a stakeholder of Raku nowadays, right - this seems to be a perpetual disclaimer. But I'm not a hater. I don't wish anybody to be "punished", I rather wish the product to improve and therefore the people who have an impact to do better.

The language does have distinguishing qualities that have potential, whether it's the insane abundance of grammar, or literally the grammar (parser) feature, or the regex with more pleasant syntax and semantics, the often cited "sane-by-default" math, Unicode (and grapheme) native string handling, concurrency-and-parallelism primitives, a massive core library with great support for multiple paradigms, etc... and the convenience of Perl-ish shell scripting utilities, showing the origins of the language.

Don't waste potential on marketing features that really do seem experimental: error-prone, resource-heavy, design-heavy and undefensible in the long run. The time of trying to sell (failed) experiments is long gone. PHP has shown in the last decade that a language can admit and move away from dead ends without abolishing its ethos and identity. Hell, even Perl can have nice things these days! There is more than enough cheap sensationist nonsubstantial Raku slander out there - the least you owe the people who work their way through that is honesty and a way forward. With regards to moving away from Junctions, this is not a technical challenge (especially now) but a choice to make - much like with Allomorphs.

Anyway, if this article only achieves as much as opening a discussion about Junctions and how they can be used well, if at all - a discussion that goes beyond the clichés of "look how cool this trivially simple code snippet works" - it's already worth it.