Raghu Bharadwaj

Posted on Sep 1 Originally published at techveda.live

Kernel & Embedded News: Linux 7.3-rc1 Out; NVMe 2.4 Goes Post-Quantum

#linux #kernel #embedded #storage

Linux 7.3-rc1 shipped on 30 August, closing the second-busiest merge window in kernel history at 15,267 commits and removing two ancient filesystems and a set of unmaintained drivers. This edition covers what the close of the window settles: per-device UBI wear-leveling thresholds for NAND devices, sched_ext sub-scheduler support reaching feature complete, Qualcomm's plan to make its Modular acquisition an open, vendor-neutral AI software layer, and the NVMe 2.4 specification's post-quantum and power-control additions.

The common thread this week is consolidation. The kernel's 7.3 picture is now fixed and can be planned against, and two industry moves — one in AI software, one in storage specifications — aim to reduce how much of a product's software stack has to be rebuilt when the hardware underneath it changes.

In this edition

Linux 7.3-rc1 closes the second-busiest merge window ever — 15,267 commits, a near-41-million-line tree, and a wave of removals now locked in for an October release. — plan your test window
UBI gains per-device wear-leveling thresholds — the threshold stops being one compile-time constant for every NAND device in the system. — evaluate for mixed-flash designs
sched_ext sub-scheduler support is feature complete — a root BPF scheduler can now hand a cgroup subtree to a nested scheduler with revocable CPU grants. — long-term watch
Qualcomm bets on an open AI software stack — Modular's Mojo and MAX aim to make accelerator choice a hardware decision rather than a software rewrite. — planning signal
NVMe 2.4 brings post-quantum security and power controls — the specification set published in August reshapes what storage procurement will ask for. — roadmap input

Linux 7.3-rc1 closes the second-busiest merge window ever

Linus Torvalds released 7.3-rc1 on 30 August and closed the merge window. The numbers are unusual: 15,267 commits were pulled in two weeks, a total only the 6.7 cycle has exceeded, and the tree now stands at 40.98 million lines. About a third of the rc1 patch is one item — AMD's DCN6 display register headers and the code for its next graphics generation — which alone pushed the AMD graphics driver directory to 6.52 million lines, roughly 16 percent of the entire kernel.

We covered the first week of this window as it happened: the 32-bit Arm platform deprecations, and the crypto and AF_ALG changes in the 25 August edition. The second week settled the removals. The EFS filesystem, the read-only driver for SGI's pre-XFS IRIX format, is gone after more than twenty years without a maintainer. The freevxfs driver for Veritas VxFS is gone too; Christian Brauner's removal note records one known user in fifteen years. The SGI XP and GRU drivers, the IBM RSA management driver, the IPWireless PCMCIA driver and the old Moxa serial drivers were also dropped. Moxa PCIe multi-port serial boards, widely used in industrial systems, are not orphaned by this: they get a replacement in the new 8250_mxpcie driver.

The removals have a counterpart on the additions side. The window that deleted EFS and freevxfs — about 3,700 lines between them — added FailFS, a 931-line pseudo-filesystem from Christian Brauner in which every operation fails with EOPNOTSUPP. Paired with a new fchroot() system call and an FD_FAILFS_ROOT sentinel, it lets a process shed its filesystem state completely: with its root in FailFS, every path lookup must be anchored at an explicit file descriptor, and absolute paths, absolute symlinks and AT_FDCWD-relative lookups simply fail. For anyone building sandboxed services, that is a cleaner primitive than the chroot-to-empty-directory conventions it replaces, and unprivileged processes can use it under no-new-privileges.

Torvalds attributed part of the volume to AI coding agents, which are both finding bugs and generating patches at a rate maintainers have not seen before; the networking maintainers used their pull request to say plainly that the volume of machine-generated reports and patches has overwhelmed their review capacity. This is now a structural property of kernel development rather than a one-cycle anomaly, and it showed up in this window both as extra fixes and as extra deletions — several of the removed drivers went because they had become permanent targets for automated bug reports nobody could act on.

On the filesystem side, David Sterba's Btrfs pull brought measured wins. Direct I/O moved onto the IOmap bounce buffer, lifting it from roughly half of theoretical throughput to around 95 percent; replacing an XArray with a local LRU list for tracking inhibited extent buffers yielded around 3x in that path; removing an unnecessary one-jiffy delay in the non-SSD mode with multiple logging tasks raised throughput around 5x on a sample workload; and full fsync of files with many extents but no holes gained a similar factor by skipping hole detection. The old free-space cache v1 code is also now disabled by default, completing a transition that began years ago. A six-week stabilisation series follows, with the release expected in the second half of October.

What it means for engineering teams

Treat rc1 as the feature freeze it is. If your product tracks mainline or you rebase a vendor kernel forward once a year, the 7.3 contents are now fixed enough to plan against: check whether any of the removed drivers or filesystems appear in your defconfig, and schedule a boot test of an rc on your reference hardware before October rather than after. Removals of this kind never announce themselves twice — the next signal you get is a build failure.

A concrete check takes minutes. Grep your defconfigs and any fragment files for the removed symbols — EFS_FS, VXFS_FS, the SGI XP and GRU options, IBMASM — and for out-of-tree modules that touch the old Moxa serial drivers, which now map to 8250_mxpcie. Teams running Btrfs on build servers should also plan a before-and-after benchmark of their own workload during the rc series; the direct I/O and fsync changes are the kind that show up in CI wall-clock times.

UBI gains per-device wear-leveling thresholds

UBI keeps an erase counter for every physical eraseblock it manages. When the difference between the highest and lowest counter on a device exceeds a threshold, the wear-leveling worker starts moving data from little-erased blocks to heavily-erased ones, evening out wear across the flash. Until this cycle that threshold was a single compile-time constant, CONFIG_MTD_UBI_WL_THRESHOLD, default 4096, applied to every UBI device in the system. Merged for 7.3 is work by Ran Hongyun of Huawei that makes the threshold configurable per device at attach time, through a new wl_threshold option on the ubi.mtd module parameter, with a value of zero falling back to the compile-time constant.

The kernel's own configuration help has documented the problem with a single value for years: the 4096 default is sized for SLC NAND and NOR with eraseblock endurance of 100,000 cycles or more, while MLC parts with endurance under 10,000 cycles want something like 128 or 256. A system could always pick one value or the other — by recompiling — but never both at once.

Our UBI on raw NAND piece two days before this landed described wear-leveling as one of the two jobs UBI exists to do. This change addresses the case the single constant handled badly: a system carrying two NAND parts with different erase endurance — say an SLC boot flash beside a cheaper MLC data flash. One threshold tuned for the SLC part lets the MLC part accumulate a wear spread that is a large fraction of its whole life before leveling begins; one tuned for the MLC part makes the SLC device churn data it did not need to move, adding erase cycles and I/O load for nothing.

There is a second, quieter benefit for product families. Because the threshold was a Kconfig constant, board variants with different flash parts previously needed either separate kernel builds or a compromise value; a boot-time per-device option lets one kernel image serve every variant, with the difference expressed in the bootloader arguments alongside the rest of the board configuration.

How to evaluate it for your device

This matters at design time, not as a retrofit. If your board carries a single NAND part, the compile-time default remains fine and there is nothing to do. For mixed-flash designs, work out the endurance ratings of each part from the datasheets and set the boot-time threshold per device accordingly once you are on a 7.3-based kernel — the endurance figures in the flash datasheet, not habit, should set the number. The option arrives with 7.3, so for products on 6.12 or 6.18 LTS this is a note for the next platform revision rather than something to backport. One caution: the exact argument syntax should be read from the merged kernel documentation when you adopt it, since attach-time options have accumulated positional fields over the years and getting the field order wrong attaches the device with defaults silently.

sched_ext sub-scheduler support is feature complete

The sched_ext framework lets a BPF program replace the kernel's scheduling policy. Over the last three cycles it has been growing a hierarchy: 7.1 made the dispatch path hierarchical, 7.2 added the supporting infrastructure, and the enqueue path was still pending when we published our hands-on guide to writing a minimal scheduler last month. The 7.3 pull from maintainer Tejun Heo completes that path. Sub-scheduler support is now feature complete: a root BPF scheduler can hand a cgroup subtree to a nested sub-scheduler along with revocable CPU grants, and the sub-scheduler owns all scheduling decisions for its tasks on those CPUs.

The word "completes" is doing precise work here. The 7.1 and 7.2 cycles could delegate only dispatching — a sub-scheduler could choose which queued task ran next, but the parent still controlled how tasks entered the queues, when one task could preempt another, and CPU frequency selection. With 7.3, parent schedulers grant and revoke all of those per-CPU capabilities, and the grants are enforced on every path a scheduler can reach a CPU through, not just the polite ones. The in-tree scx_qmap example now demonstrates the full hierarchy, which gives anyone building on this a working reference.

The robustness work is as interesting as the delegation. A task whose scheduler has no access to the CPUs it needs used to starve until the watchdog ejected the entire scheduler; the kernel now runs such tasks directly on a small bandwidth budget, converting a scheduler-killing failure into bounded degradation. Tasks migrating across a sub-scheduler boundary are now re-homed to the new owner, closing both a wrong-scheduler-scheduling bug and a use-after-free. The abort path became NMI-safe, arena objects now cross the kernel/BPF boundary as typed pointers instead of manually translated untyped ones, and BPF-writable arena memory is validated before the kernel uses it — with explicit synchronisation rules for task slice and vtime writes — closing corruption paths open to buggy or hostile schedulers.

One further detail matters for LTS users: the pull also carries core scheduling fixes that missed 7.2, including one where interleaved core-wide task selections could corrupt each other's state and hang the machine, and those fixes are marked for stable. They will reach the LTS branches through the normal stable process even if you never run a BPF scheduler.

What it means for scheduler work

Hierarchical scheduling is the piece that makes sched_ext credible for mixed systems — an infotainment stack and a telemetry stack on one SoC, each under a scheduler tuned for it, without either being able to damage the other. The failure-containment work matters as much as the delegation: a sub-scheduler bug now degrades its own subtree instead of taking down scheduling machine-wide, which is the property a safety argument or a multi-tenant design actually needs. If you experimented earlier and stepped back, 7.3 is the version where the sub-scheduler API stops moving.

Prerequisites are unchanged: the kernel needs CONFIG_SCHED_CLASS_EXT and a BPF stack with CONFIG_DEBUG_INFO_BTF, the second of which is the one most often missing from custom embedded kernels. Confirm the framework on your kernel before planning anything:

root@rock-5b:~# cat /sys/kernel/sched_ext/state
enabled
root@rock-5b:~# cat /sys/kernel/sched_ext/root/ops
simple

Qualcomm bets on an open AI software stack

Qualcomm completed its acquisition of Modular, the AI software company co-founded by compiler engineer Chris Lattner, in July. Details of the plan emerged last week, and they are worth the attention of anyone who has done an accelerator bring-up. Modular's stack — the Mojo language, now at 1.0 with its compiler and toolchain under the Apache 2.0 licence, and the MAX inference framework — aims to let the same model code run across GPUs, TPUs and custom accelerators from different vendors. On Nvidia hardware it replaces the CUDA math libraries and kernels wholesale, keeping only a small device-management layer.

Qualcomm's own framing, from Rashid Attar, its head of data center engineering, is that a challenger vendor has two problems to solve in sequence: it must deliver differentiated hardware, and then it must make that hardware usable without the customer assigning engineers to port existing workloads — because, in his account, customers say they will assign exactly none. Modular is the attempt to remove the second problem. The reason it is hard is that the incumbent's advantage is no longer just CUDA: industry analysts point out that the software moat now extends across interconnects, data-processing units, libraries and integrated rack-scale systems, so a CUDA translation layer on its own no longer changes the decision.

The central claim is about bring-up cost. In Modular's own account of porting to AMD's MI355X accelerator, two engineers brought the stack up in fourteen days, with 99.9 percent of the code architecture-agnostic; the hardware-specific remainder concerned new BF16 conversion instructions, larger tensor-core tiles and increased shared memory. The result, Modular says, outperformed AMD's optimised vLLM fork by up to 2.2 times on the workloads tested. Those are the vendor's own figures and should be read as such. But the structural argument stands independently: if the portable layer is thick and the per-silicon layer thin, new silicon competes on merit rather than on the size of its software ecosystem — and the engineering effort concentrates on the small part of the stack that actually changed.

The obvious tension is that a neutrality layer is now owned by one of the vendors it arbitrates between. The mechanisms offered are concrete but partial: Mojo's compiler and toolchain sit under Apache 2.0, which Lattner describes as an irrevocable grant that cannot be withdrawn later; Attar describes organisational firewalls under which Modular engineers may see confidential data about competitors' hardware that Qualcomm's own accelerator teams will not; and an industry alliance giving other hardware companies a governance role is planned for later this year. Lattner's own summary of the neutrality question was that there is no perfect answer to it. The honest reading is that neutrality clearly serves Qualcomm while it is the challenger; the test comes if it stops being one.

How to read it as a planning signal

For teams selecting edge or datacenter AI hardware, the question to track is whether the per-silicon bring-up layer really stays thin for parts beyond the large vendors. If it does, smaller NPU and accelerator vendors — which cannot afford to build a CUDA-class software ecosystem of their own — become viable choices, and hardware selection shifts toward performance per watt, availability and price. There is also a caution in the other direction: teams for whom every microsecond matters will keep optimising against one architecture directly, and an abstraction layer that hides hardware complexity can hide hardware advantages too. Watch the alliance's membership and licence terms when it launches; that list will say more than the benchmarks do.

NVMe 2.4 brings post-quantum security and power controls

NVM Express published the NVMe 2.4 specification set on 4 August — earlier than this edition's window, but a development we had not covered and one with a long tail. The release also restructured the specification family itself, splitting it so that individual capabilities can be revised without republishing the whole base document; the stated aim is faster, simpler development of the specifications, which in practice means features will now arrive more continuously rather than in large multi-year revisions.

The 2.4 feature set groups into three themes. Security: post-quantum cryptographic algorithms in the host-to-SSD data path, TLS 1.3 transport security for NVMe over fabrics, and Key-Per-I/O encryption, which allows a different key per individual I/O command rather than per namespace — the granularity multi-tenant systems have been asking for. Manageability: configuration recovery, and a new virtualization architecture that supports live migration of VMs using locally attached SSDs, removing one of the last reasons cloud operators avoided direct-attached NVMe for migratable workloads. Power and sustainability: controller-based performance controls and voltage monitoring, giving the host standardised control over a drive's draw. Updates to Zoned Namespaces, Key-Value, rotational media and endurance-group management are also included.

Two of these matter directly to embedded and edge builders. The power and performance control features are relevant wherever storage sits inside a fixed thermal budget, which describes most fanless industrial and edge-AI boxes — today that constraint is handled with vendor-specific tooling or by over-provisioning the enclosure, and a standard host-side interface changes both. And the post-quantum additions will flow into procurement: regulated buyers are already writing quantum-resistant requirements into tenders, and drives claiming 2.4 compliance will become the easy way to satisfy them.

What it means for device builders

Nothing ships tomorrow; specification revisions reach real drives over a couple of product generations. The near-term action is procurement language: when specifying storage for a product with a seven-to-ten-year life, ask vendors for their NVMe 2.4 roadmap, and treat post-quantum support in the data path as a feature that will be demanded mid-life even if no one is asking today. Remember also that specification features are individually optional — "NVMe 2.4 compliant" on a datasheet does not by itself mean the drive implements Key-Per-I/O or the power controls, so procurement questions should name the specific features, not the revision number. For Linux support of the new features, watch the NVMe driver changes in the next two or three kernel cycles.

References

— Raghu Bharadwaj

If you work on the Linux kernel or embedded Linux and want structured, instructor-led depth on these subsystems, see TECH VEDA's training programs.

DEV Community