GPT-5 and Convex Optimization: What the Claims Actually Mean for Engineering Tooling

A recent thread hit 482 points and 312 comments on the claim that GPT-5 has made meaningful progress on convex optimization problems that have been largely intractable for thirty years. The reaction split cleanly: researchers excited about specific benchmark results, engineers skeptical about practical implications, and a third camp arguing the framing was misleading from the start.

All three groups have valid points. Here's an attempt to untangle them.

What Convex Optimization Actually Is (And Why It's Everywhere)

Convex optimization is the class of problems where you minimize a convex function over a convex set. The key property: any local minimum is a global minimum. That makes convex problems tractable in a way that general optimization problems are not.

This shows up constantly in engineering infrastructure:

• ML training — loss surfaces are often non-convex, but large subproblems (SVM training, certain regularization schemes) are convex and solved with specialized solvers like CVXPY or Gurobi
• Compiler backends — register allocation and instruction scheduling have convex relaxations that inform heuristics
• Resource allocation — cloud schedulers, network routing, and capacity planning often reduce to linear or quadratic programs
• Financial modeling — portfolio optimization under constraints is a textbook convex problem

The "30-year gap" framing refers to open problems around scalability and solver efficiency — specifically, the gap between what interior-point methods can handle in theory and what's practical at production scale.

Solvers like MOSEK and SCS are excellent. They're also slow at scale, sensitive to problem conditioning, and require expert formulation. That last point is the friction that actually matters for engineering teams.

What the GPT-5 Results Actually Show

The specific claims center on GPT-5's ability to:

1. Correctly formulate optimization problems from natural-language descriptions
2. Identify when a problem has convex structure that allows efficient solving
3. Generate CVXPY or similar solver code that compiles and runs correctly
4. In some benchmarks, suggest reformulations that meaningfully improve solver performance

This is not "GPT-5 solved convex optimization." It's closer to: GPT-5 has become a competent co-pilot for the formulation step, which is where most engineering time actually goes.

To make this concrete — if you're building a job scheduler and need to minimize total latency subject to resource constraints, writing that as a proper LP or QP has always required knowing the vocabulary: decision variables, objective function, constraint matrices. That knowledge barrier is real. Most backend engineers reach for heuristics not because an optimal formulation wouldn't be better, but because they don't want to spend three days reading Boyd and Vandenberghe.

# What you used to need domain knowledge to write:
import cvxpy as cp
import numpy as np

# Job scheduling: minimize makespan across n jobs, m machines
n_jobs, n_machines = 10, 3
processing_times = np.random.randint(1, 10, (n_jobs, n_machines))

x = cp.Variable((n_jobs, n_machines), boolean=True)
makespan = cp.Variable()

constraints = [
    cp.sum(x, axis=1) == 1,  # each job assigned to exactly one machine
    makespan >= cp.sum(cp.multiply(processing_times, x), axis=0)
]

objective = cp.Minimize(makespan)
problem = cp.Problem(objective, constraints)
problem.solve()

GPT-5 can now generate that correctly from a plain-English description of the scheduling problem. More importantly, it can flag when you've accidentally written a non-convex constraint that will cause the solver to fail silently or return garbage.

The Skepticism in the Thread Is Legitimate

The 312-comment thread wasn't just noise. The pushback clustered around several real concerns.

"Benchmark performance doesn't transfer to production problems." True. The benchmarks use well-conditioned, textbook-scale problems. Real scheduling problems have messy side constraints, integer variables, and numerical conditioning issues that stress even expert formulations. GPT-5 generating plausible-looking CVXPY code is not the same as that code solving your actual problem.

"This conflates formulation assistance with algorithmic progress." Also true. The headline "closing a 30-year gap" implies progress on solver algorithms — faster interior-point methods, better preconditioning, novel dual decompositions. That's not what's being described. What's being described is better tooling around using existing solvers. Valuable, but a different category of claim.

"LLMs hallucinate constraints." This is the one that should give you pause before deploying any of this. GPT-4 can generate convex-looking constraint sets that are subtly non-convex — the kind of error that doesn't throw an exception, but silently returns a wrong answer. GPT-5 is better at catching these, but "better" doesn't mean "safe to run unsupervised in a production resource allocator."

The honest framing: this is a significant improvement in the accessibility of optimization tooling, not a breakthrough in optimization theory.

What This Actually Changes for Engineering Teams

If you're building backend infrastructure, developer tooling, or any system that currently uses heuristics because "setting up a proper solver seemed like too much work," the calculus has shifted.

The formulation barrier is lower. You can describe your problem in prose, get a working CVXPY sketch, and iterate from there. That's genuinely useful even if you still need to validate the output carefully.

Debugging solver failures is faster. Infeasibility and numerical issues in optimization problems have always been hard to diagnose. Having a model that can look at your problem and say "this constraint is likely causing infeasibility because X" cuts debugging time substantially.

The integration surface with existing infrastructure is unchanged. CVXPY still calls MOSEK or SCS. Solver performance characteristics are the same. If your problem is too large for current solvers, GPT-5 doesn't fix that. You still need decomposition, approximation, or a different approach.

For teams building scheduling systems, ML infrastructure, or resource allocation layers — say, a Node.js/TypeScript backend calling out to a Python optimization service — the practical win is in prototyping speed and reducing the expertise required to reach a working formulation.

// A backend service that now has a shorter path to proper optimization
// instead of a hand-rolled heuristic
async function scheduleJobs(jobs: Job[], machines: Machine[]): Promise<Assignment[]> {
  // Previously: greedy heuristic because writing the LP felt like too much
  // Now: call a Python optimization service with a properly formulated problem
  // generated and validated with AI assistance
  const formulation = await optimizationService.solve({
    type: 'linear_program',
    objective: 'minimize_makespan',
    jobs,
    machines,
  });
  return formulation.assignments;
}

Where to Be Careful

A few concrete cautions before building this into critical infrastructure.

Always verify the problem class. If the model says your problem is convex, check it. A non-convex problem handed to a convex solver will either fail noisily or return a locally optimal point far from the global optimum. In a scheduler or resource allocator, that has real consequences.

Numerical conditioning still requires human attention. LLMs generate well-scaled constraint matrices for textbook problems. Your production problem has characteristics the model hasn't seen. Check condition numbers. Know your solver's tolerance settings.

Don't skip domain expert review. The formulation step is where domain knowledge matters most — not just mathematical correctness, but whether the optimization objective actually captures what you care about. GPT-5 can give you a mathematically valid LP that optimizes the wrong thing because it misunderstood a business constraint.

The Actual Opportunity

The real engineering opportunity here isn't "replace your solver with GPT-5." It's closer to: a class of infrastructure problems previously gated behind specialized knowledge is now more accessible to generalist engineers.

That has compounding effects. Teams that previously used greedy heuristics because the optimization setup cost was too high can now prototype proper formulations. Those prototypes, even if they need expert refinement, start from a better place than a hand-rolled approximation.

For developer tooling specifically — compilers, build systems, deployment schedulers — this opens up experimentation with optimization-based approaches that teams routinely deprioritized because of implementation cost.

The 30-year gap in optimization theory hasn't closed. But the gap between optimization theory and something an engineering team can actually use got meaningfully smaller. That's a more modest claim than the headline suggested — but it's a real one, and it's worth taking seriously.

Comments

[Mike Czerwinski]

The reframe from breakthrough to accessibility is the right correction, and it has a sharper edge if you push on what accessibility actually removes. The hard part of convex optimization was never typing the CVXPY, it was knowing whether the problem you typed is the problem you have. GPT-5 automates the part that was already checkable, does the code run, and leaves the part that was never checkable by running, is this formulation faithful, exactly where it was. Except the knowledge that let you write the formulation was the same knowledge that let you know it was right, so removing it as a prerequisite removes it as a check. The barrier was also the filter.

Which is why the silent-wrong-answer worry is the real one, and it is worth being precise about where it lives. Non-convexity itself is not the silent case, DCP catches that, CVXPY rejects a formulation that does not compose to a convex program. The silent case is the model making the problem convex by quietly changing it: relaxing a side constraint, dropping an integer requirement, approximating a nasty term. The code runs, the program genuinely is convex, DCP passes, the solver returns a clean global optimum, and it is the global optimum of a different problem than the one you asked. The convexity is real. The fidelity is what is gone, and nothing in the solve path checks fidelity.

The useful part is that this is one of the rare cases where the invisible claim has a cheap oracle, and it comes from the same math the post is about. A convex relaxation always bounds the original problem, so if the convex formulation is faithful its optimum has to sit close to what a general solver or even a sampling baseline finds on the original, unrelaxed problem. Solve it both ways and gate on the gap. A large gap means the convex version is optimizing something the original did not ask for. So the tooling that lowers the formulation barrier should ship with that differential check attached, precisely because the users it newly enables are the ones who removed their own ability to audit the relaxation by hand. Accessibility is fine as long as it carries its own filter, since it dissolved the one that used to come bundled with the knowledge.