Everyone benchmarks the algorithm. Almost nobody talks about the layer that actually breaks projects: modeling.
Route optimization projects rarely fail on the algorithm. Between a business rule and a routed plan sit two other layers: whether the modeling language can express the rule at all, and whether someone stated it correctly.
The first is modeling distance, the gap between how a rule exists in an operation and how a formalism lets you write it down. It's an engineering problem, and it's countable: 61 abstract primitives producing 149 sector variants in our library as of August 2026. The second is human, and it gives no signal when it fails. It's where most of the last ten years of customer projects went, and it's what still stands between route optimization and a self-serve product.
A delivery time window is a business constraint. In almost every routing API it's also an object: set a start, set an end, done. No modeling effort at all.
A minimum daily revenue below which a subcontractor refuses a route is also a business constraint. It's an object nowhere. It has to be built from whatever the formalism offers, assuming it offers enough, and assuming somebody realized the rule existed.
Both bind the plan equally. The distance between them is the subject of this article, and it has nothing to do with the algorithm.
The layer everyone benchmarks is the one that rarely fails
Solvers get compared on speed, on gap to optimality (how far a solution sits from the mathematical best), on stops absorbed. Those comparisons are legitimate, and we run them on our own engine. They're also the wrong place to look for why a project failed.
The algorithmic problem is hard, and it's well served. Vehicle routing has been studied for sixty years, metaheuristics (general-purpose search algorithms) work, and a competent commercial solver beats experienced planners on distance or vehicle count by a wide margin. We've made that argument ourselves. That success created the blind spot: gains that large made the algorithm look like the whole product.
In ten years we haven't seen a project fail because the solver was too slow. We've seen plenty fail because the plan described an operation that didn't exist.
Three layers sit between a business rule and a solver. Only one is the algorithm.
Getting from a rule that exists in an operation to a plan that respects it takes three things:
- Expressivity — Can this rule be stated in the modeling language at all? Engineering problem, inside the solver.
- Formulation — Has it been stated correctly, with the right hardness and scope? Human problem, between operations and mathematics.
- Search — Given a correct statement, how good a solution, how fast? Algorithmic problem.
The industry benchmarks the third layer. Projects fail in the first two.
The upper layers get collapsed into one, and the time window example shows why: when a formalism has a native object for a rule, expressing it and expressing it correctly happen in the same gesture. They come apart as soon as the rule has no native object.
They also fail differently. When you hit an expressivity limit, you notice fast: you look for a way to state your rule, you don't find one, and you know within the hour. A formulation error hides better. You state the rule easily, you state it wrong, and nothing signals it.
A third case sits between the two, and it's the one that misleads. The primitive that carries your rule exists, but it's abstract enough that the mapping isn't obvious, so the developer concludes the API can't do it. That's a translation failure dressed as an expressivity limit. It usually ends in a workaround built outside the model, where the solver can't see it and can't optimize against it.
Why a time window is free and a subcontractor floor is not
Modeling distance is the gap between how a rule exists in an operation and how the formalism lets you write it down. It's a property of the formalism, not of the rule. The same constraint carries a different modeling distance in different tools.
- Zero distance — the rule has a native object. A time window in a modern routing API: two fields, nothing to translate.
- Moderate distance — the rule has no native object, but the formalism offers abstract constructs that can carry it. Most real constraints live here, and the subcontractor floor is one of them. No routing API on the market exposes it as an object, ours included, so it has to be assembled from abstract primitives. We tested the rule across eleven APIs. Three could express it, each in its own way. For the other eight, we found no way to state it at all. Possibly one exists and we missed it, which is exactly the third case above.
- Maximal distance — the formalism is a general mathematical language and every constraint is an engineering project. Writing a rich vehicle routing problem directly in mixed integer programming is the reference case [5]. Each rule costs days.
Reducing modeling distance is a product decision. It determines how much work lands on developers.
61 primitives, 149 sector variants: what expressivity actually costs to build
Reducing modeling distance means turning constraints that would be engineering projects into native constructs. That's a solver problem, and it's the central engineering challenge of any optimization engine whatever its domain or solving technique [5]. Adding expressivity isn't the hard part; adding it without degrading solver performance is. Every construct you make native is one the engine has to handle efficiently inside the search, which is likely why eight of the eleven APIs we tested can't express the subcontractor floor. Not an oversight on their side, but a trade they made.
As of August 2026, Kardinal's library holds 61 abstract primitives in 11 categories, producing 149 concrete constraints across the 7 sectors we work in. Those are the ones we've encountered. It's an inventory of what ten years of projects surfaced, not a claim about the boundaries of the problem.
The duplication is deliberate: one formalism carries different operational meanings by sector. Maximizing preferential assignment between stops and resources means matching a driver to their home zone in parcel delivery, and matching a junior technician to easy interventions in field service. Same mathematics, different conversation, different ways of getting it wrong.
Which is why a new business rule rarely means a new primitive. Constraints we've never seen before are the minority case, and when they do appear, most turn out to be expressible with primitives that already exist.
Expressivity is also the one layer an outsider can compare, using nothing but public documentation. We've been publishing that comparison since 2024, and we know of no equivalent from anyone else. Our 2026 benchmark of eleven routing APIs tests them against a constraint list we rebuilt from scratch. We're not a neutral party in it, which is why the constraint list and the scoring method are published rather than summarized.
A perfect formalism still doesn't make the model correct
There's a tempting conclusion here: keep pushing expressivity and the modeling problem disappears. Give every rule a native object and nothing is left to translate.
The time window shows why not. The object is native, the modeling distance is zero, and these questions are all still open. Is the window hard or penalized? Does it apply to arrival or end of service? Is a five-minute overrun a failure or a degradation? Does it hold when the customer is absent? Each answer changes the plan, and none of them gets easier just because the object is well-defined.
Expressivity lowers the cost of translation. It rarely lowers the cost of specification.
Operations research named this failure long before routing software existed: the Type III error Kimball described in 1957, giving the right answer to the wrong problem [1]. How a problem gets framed in the first place is among the strongest determinants of the solution you end up with [2].
Our subcontractor case is exactly that. A shipper pays an internal fleet per kilometer and subcontractors per task. Optimize that cost model and the solver rationally hands dense routes to the internal fleet and thin ones to subcontractors. The plan is optimal and unrunnable, because a subcontractor still pays for a full driver shift and declines anything below a certain daily revenue. That floor was written down nowhere. Planners had applied it for years without ever stating it. No amount of expressivity would have helped, because nobody had asked the question.
Who can actually do this, and why it doesn't scale
Formulation requires holding two things at once: how the operation actually runs, and what the formalism will do with a given statement of it. The same need can be modeled several ways with different side effects, and knowing which one produces which behavior is the entire skill.
That person sits in one of two places. Inside the customer, an operations profile with an unusually analytical turn of mind, rare enough that we don't plan around it. Or inside the vendor.
We built the second. Our solution engineers do this work and they're why our projects converge. They're also a bottleneck hiring doesn't remove: the profile is rare, and every project consumes one of them from first workshop to production. That's workable for enterprise projects. It's impossible for a self-serve API.
Which leaves this category with one open question, and it's not an algorithmic one. The expertise that turns an operation into a correct formulation exists, but it's written down almost nowhere a machine can read. Today it only exists inside the people who do this work project after project.
Curious how others here have dealt with this. If you've built domain expertise into a system, an agent, a rules engine, whatever, how did you get that tacit knowledge out of someone's head and into something a machine could actually use? Did it work, or did you hit the same wall we did?
Sources
- Kimball, A. W., Errors of the Third Kind in Statistical Consulting, Journal of the American Statistical Association, 52(278), 1957, pp. 133–142.
- Mitroff, I. I. and Featheringham, T. R., On Systemic Problem Solving and the Error of the Third Kind, Behavioral Science, 19(6), 1974, pp. 383–393.
- Caceres-Cruz, J. et al., Rich Vehicle Routing Problem: Survey, ACM Computing Surveys, 47(2), 2014.
- Drexl, M., Rich Vehicle Routing in Theory and Practice, Technical Report LM-2011-04, Johannes Gutenberg University Mainz, 2011.
- A Review of Approaches to Modeling Applied Vehicle Routing Problems, arXiv:2105.10950, 2021.
Top comments (7)
The line I would push on is that formulation gives no signal when it fails. I think it gives a good one, and it is usually sitting in a table nobody joined.
The subcontractor floor was enforced for years before anyone wrote it down, and enforcement leaves a record. Every route a subcontractor declined, renegotiated or quietly handed back is a labelled point on the wrong side of a boundary nobody stated. Same for the planner who reassigns stops after the plan lands. On the parcel side of enterprise logistics that was the most informative data we had and the least used, because the plan lived in one system and what actually got run lived in another, and nobody owned the difference between them.
So the detector for a formulation error is the delta between the plan you emitted and the plan that was dispatched, read per segment rather than in aggregate. Scattered deviation is weather and traffic. Deviation that is systematic, in one direction, on one class of route, is a constraint you did not model. You do not need to know what the rule is in order to know that there is one, and that is what makes it cheap: it is a monitoring problem before it is a knowledge extraction problem.
On getting it out of someone's head, I think interviews stall because you are asking for a rule and people do not hold rules. They hold judgments. Nobody can enumerate what makes a route unacceptable, and almost anyone can look at one and say no. What worked was showing candidate plans and collecting accept or reject, then moving one number until the answer flipped. "Would you have run this one at 340 a day?" gets answered in four seconds. "What are your constraints?" does not get answered at all, and the honest people will tell you as much.
The caution is that an override log gives you boundaries rather than rules. A planner pulling three stops off a subcontractor route might be enforcing the revenue floor, protecting a customer relationship, or working around a driver nobody wants to name out loud. The log says where the boundary sits. Only the flip test says which rule drew it. Which might narrow your self-serve problem rather than solve it: the part your solution engineers are irreplaceable for looks less like the modeling and more like that disambiguation step, and those are different things to try to automate.
Fair push, and I was too absolute. Enforcement leaves a record. The honest version is that the signal goes unread, not that it doesn't exist (to be honest, most of our users just don't have that data, therefore in practice the signal is actually inexistent in most cases).
What I'd still defend: the record gives you a boundary, and the flip test asks the person whose judgment drew it. Someone who has enforced a habit for five years answers fast, consistently, at the same number every time. That proves a stable rule exists, not that it's load-bearing. Telling a constraint from a habit needs a referent outside the system, a contract or a cost model or a controlled test. The delta doesn't carry one and neither does the flip test.
The 2021 Amazon Last Mile challenge is the cleanest public case I know. Thousands of historical routes released specifically to learn driver deviations, and the winning entry read zone order and precedence out of the data by hand, then encoded five or six constraints as penalties. The signal was there, which is your point. Turning it into a model stayed an expert step, which is mine.
Scope rather than disagreement on the detector: it needs a stable baseline to read the delta against. Pre-assigned zones give you one, since the day's route is a variation on a template. That holds much less where the plan is meant to reorganize daily, because the deviation and the intent become the same object, and not at all at day zero.
And if the boundary is real, what's still open is its hardness and its scope, which is where the money is. Enforcing driver consistency on three quarters of customers instead of all of them takes the cost penalty from around 13 percent to under 3. Same operation, same data, and the price moves by an order of magnitude depending on how you state the rule. The override log doesn't contain that choice.
I like your last paragraph, and I think you're right. We have been sketching a calibration API endpoint along those lines: historical routes in, systematic deviations from what the model would have produced out, as candidates rules for operations to look at. What we don't have is enough input from operations mature enough to have both sides of the data, so I'm genuinely unsure whether it's a real need or a thing that sounds good in a design doc. You've clearly lived it. Would you actually run something like that against your own history, and if you would, what does it have to return to be worth the integration? Or is getting plan and dispatch out of two systems that don't talk the thing that kills it before anyone gets to the modeling question?
Taking the second question first, because it decides the first one.
Yes, mostly, and not for the reason people expect. The data is usually there. What is missing is the join. The plan is keyed by planned stop and the dispatch record is keyed by scan event, and the thing that happened in between is what destroys the correspondence. A stop moved from one route to another belongs cleanly to neither key. So the integration cost is not two exports, it is reconstructing which planned stop each scan belongs to, and that reconstruction is itself a model with its own error rate. That is where these projects stall, and it sits upstream of every modelling question you would want to ask.
Which changes what your endpoint has to do first. Its matching error is correlated with its signal. Reassigned stops are both the hardest to match and the only ones carrying the deviation you are looking for, so low coverage does not merely weaken the output, it biases it toward the routes that stayed put. The first thing the endpoint should return is what fraction of the history it could match, and it should refuse to emit candidate rules below a threshold you pick in advance. An honest refusal there buys more trust than a rule list would.
On what it has to return to be worth integrating. Not a rule in words. A boundary with the routes sitting on each side of it, because an operations person cannot falsify a sentence and can absolutely look at three routes they remember and tell you which one is wrong.
And a count of how often it would have bound. You said that telling a constraint from a habit needs a referent outside the system. Frequency of binding is one, and it is already in your data. A rule that would have changed two routes a year is a habit no matter how fast and how consistently someone answers the flip test. One that binds daily is load bearing whether or not anyone can articulate it.
Your own 13 percent to under 3 number is the strongest argument for this shape. The value there was not in learning the rule, it was in learning its scope. Nobody says "three quarters of customers" in an interview. A deviation log can measure which customers actually got consistent drivers, so the output worth building is the extent of the rule as a distribution rather than the rule as a statement.
The matching point lands. Coverage correlating with the signal is the kind of thing we'd have shipped without noticing, in a tool meant to catch exactly that.
Following it further though. If a stop moves and the record gets updated in place, the original assignment wasn't lost. It was never kept. Which would mean there's no plan of record to measure against, and the real plan lived in people.
Is that a fair reading? If it is, we're pointing at the wrong thing. You can't measure deviation from something informal, you can only infer a baseline and then measure against your own inference, which will be wrong in exactly the cases you care about.
So maybe the first job isn't extraction. It's writing the plan down. Usually that's dull stuff: postcode to route pairs, GeoJSON territories, a nominal sequence per day of week, and then the per stop record kept rather than overwritten, route at emission alongside route executed. None of it is clever. But it turns the question into a lookup instead of an inference.
Your last point has the same problem from the other end. The log shows how far a rule was applied, which only helps where the operation varied. Enforce consistency everywhere and the log says everywhere.
Anyway, three replies in and you've given me more than the article did. Happy to take it off-thread if you are.
Fair reading, with one correction that makes it cheaper than you are describing.
The plan of record is not missing. It is overwritten. At emission there was a concrete assignment, the manifest that went out at start of shift, and the system then updates that row in place and destroys the record of it rather than the fact of it. So this is a write-ahead problem, not a formalization problem.
That changes what you ask for first. Postcode to route pairs, territories and a nominal sequence per weekday are a project, and it is the kind of project that stalls, because you are asking people to formalize what they experience as judgment. Keeping route at emission alongside route executed is one immutable row per stop per day and it needs nobody to agree on what the rule is. The dull item you listed last is the only one you need first.
On your second point you are right, and it is a better objection than the one I raised. Frequency of binding says nothing where the rule is enforced everywhere. But the measure I actually want is not how often the rule bound. It is how often the planner proposed something the operation then overrode. An override is only visible if you kept what was proposed, so both of your questions land on the same single change.
Happy to keep it here. It reads better in the open than it would between two people.
시간창처럼 표현 거리가 0인 제약도 도착 기준인지 서비스 종료 기준인지에 따라 계획이 달라진다는 예시가 핵심을 잘 보여주네요. 특히 계획과 실제 배차의 차이를 구간별로 관찰해 숨은 제약 후보를 찾고, 사람에게 경계값을 확인하는 절차가 모델링 실패를 줄이는 현실적인 방법으로 보입니다.
Thanks, you picked up exactly the part I care most about. The arrival vs service-completion question is a good test case: it reads as a single unambiguous constraint, and it silently splits into two different models. In our API we force that choice rather than defaulting it, because a default here is a failure nobody sees.
One caveat on the observation loop: not every plan-to-dispatch deviation is a hidden constraint. A lot of it is driver habit, local knowledge, or simple noise. That is why the boundary-value interview matters more than the deviation count. The observation gives you candidates, the human turns a candidate into a constraint with an actual value and a hard/soft status. Skipping the second step just replaces one wrong model with another.