DEV Community: alphanine

Turing Sim: Camera Ground Truth from One Hydra Render

alphanine — Mon, 19 Oct 2026 03:07:47 +0000

Camera sensors now render RGB with pixel-aligned depth, normals, object and class masks, and boxes. 相机传感器现在能渲染 RGB，并输出逐像素对齐的深度、法线、物体和类别分割以及检测框。

The overview camera switching between RGB, depth, normals, objects, classes and boxes.

One colour per object: each UR10 link, the table and the cube.

Metric depth along the optical axis, shown with an inverse-depth Turbo ramp.

Situation

After IMU and LiDAR, the sensor stage needed cameras: not only pictures, but the ground truth that robot learning uses, such as depth, segmentation and boxes, aligned exactly with the image. Turing Sim renders with Storm (rasterised) or Turing RT (ray traced) through Hydra. Neither exposed its depth or id buffers to Python.

Task

Give camera sensors every output from a single render, so all of them describe the same pixels, with accuracy that can be checked. They also had to run in the editor without degrading the viewport.

Action

Cameras are USD cameras. A camera sensor is a standard UsdGeom.Camera with turingSensor:type = "camera". Presets set the lens from a field of view and the image's aspect.
Read the AOVs back. A small native module reads Hydra's color, depth, primId and instanceId buffers and decodes ids to scene paths, including instance proxies. It works with Storm and with Turing RT.
A separate render process. Integer ids need a render without multisampling, and Hydra reads that setting once per process, so cameras render in their own worker and the editor's viewport keeps its antialiasing. Storm's eye-normal buffer turned out to clamp negative components, so normals are rebuilt from depth.
From buffers to ground truth. RGB is averaged over odd supersampling; ground truth uses each pixel's centre sample. Depth is converted to metres along the optical axis. An object is the nearest labelled prim, else its rigid body. Tight and amodal 2D boxes and oriented 3D boxes come with OpenCV intrinsics and the camera's pose.

Result

On a wall 4 m away, depth is within 2 mm wherever the wall shows, and back-projected pixels land on the wall plane within 2 mm. Storm and Turing RT agree pixel for pixel on the masks, with 0.01 mm median depth difference.

A 640×480 frame with every output takes 36 ms; with 3×3 antialiasing, 73 ms. The presets' rates follow these measurements.

The captures show the editor's camera preview switching between outputs on the UR10 workcell. A follow-up commit stopped box names from piling up on the robot's links. Frames are best effort in the editor; exact-rate capture belongs to the synthetic-data pipeline, which is next.

背景

做完 IMU 和 LiDAR，传感器阶段还需要相机：不只是画面，还有机器人学习要用的真值，比如深度、分割、检测框，而且要和画面严格对齐。Turing Sim 通过 Hydra 用 Storm（光栅化）或 Turing RT（光线追踪）渲染，两者都没有把深度和 ID 缓冲交给 Python。

目标与难点

让相机传感器从一次渲染里得到全部输出，保证所有输出描述的是同一组像素，而且精度可以检验。同时要能在编辑器里运行，不影响视口画质。

解决思路与行动

相机就是 USD 相机。 相机传感器是一个标准的 UsdGeom.Camera，带 turingSensor:type = "camera"。预设根据视场角和画面宽高比设置镜头。
读回 AOV。 一个小的原生模块读取 Hydra 的颜色、深度、primId 和 instanceId 缓冲，并把 ID 解码成场景路径（包括实例代理）。Storm 和 Turing RT 都适用。
单独的渲染进程。 整数 ID 必须在关闭多重采样的情况下渲染，而 Hydra 每个进程只读一次这个设置，所以相机在自己的 worker 进程里渲染，编辑器视口照样保留抗锯齿。另外发现 Storm 的视空间法线缓冲会把负分量截掉，所以法线改为由深度重建。
从缓冲到真值。 RGB 按奇数倍超采样取平均；真值取每个像素的中心采样。深度换算成沿光轴的米数。物体取最近的带标签 prim，没有标签就取所属刚体。输出紧框和投影框两种 2D 框、有向 3D 框，以及 OpenCV 内参和相机位姿。

结果与下一步

对 4 米外的一面墙，只要墙出现的像素，深度误差都在 2 mm 以内，反投影回去的点也落在墙面 2 mm 以内。Storm 和 Turing RT 的分割掩码逐像素一致，深度中位差 0.01 mm。

带全部输出的 640×480 一帧要 36 ms；开 3×3 抗锯齿要 73 ms。预设的帧率就是按这些实测值定的。

截图展示的是编辑器里的相机预览在 UR10 工作站上切换各种输出。后续一个 commit 修好了机器人连杆上框标签互相重叠的问题。编辑器里的帧是尽力而为；按精确帧率采集属于合成数据流程，那是下一步。

Turing Render: From Grey Fallbacks to MaterialX Glaze, Gold and Fitted Studio Light

alphanine — Mon, 19 Oct 2026 03:07:32 +0000

Real Turing RT renders: the MaterialX Teapot and Lion with their materials, and coarse meshes without black rims.

The ASWF MaterialX Teapot and Lion now render their glaze, brass, gold and wood under a studio fitted to the official images (left: official render, middle: before, right: after).

Low-poly smooth-shaded spheres lose their black silhouette rims, and the shadow terminator of a 12-segment sphere becomes smooth.

Situation

Turing RT is the Metal path tracer behind Turing Sim, our robot simulator. It already had OpenPBR-style layered materials, HDRI importance sampling and an AI denoiser, but the question of the day was blunt: why do the renders still not look good?

The first finding was about content, not the renderer. The robot workcell scenes had no lights and no materials, only display colours, so path tracing and realtime ray tracing produced nearly identical images: there was nothing to bounce off and nothing glossy to reflect. Adding a floor, two area lights and a few real materials to the same robot immediately showed soft shadows and glossy reflections. That moved the work to the cases where the renderer itself was wrong.

Task

Make glass and metal hold up on real assets, starting with a material test scene.
Support the NVIDIA MaterialX Teapot and Lion from the ASWF Digital Production Example Library, which rendered completely grey.
Explain why their colours still differed from the official renders, and close as much of that gap as could be measured.
Make Turing Sim always use the latest renderer.

The constraint throughout was the project rule that every rendering change ships with a measurement against a reference.

Action

Dark rims on smooth low-poly meshes. Metal spheres showed black rims. Swapping them for 256×128 meshes made the rims vanish, which pointed at interpolated normals on coarse facets: silhouettes were mistaken for back faces and reflections went into the surface. The fix decides front and back from the face, keeps opaque shading normals valid (the approach Cycles uses, with its quadratic root checked numerically over 200,000 random cases), and lifts shadow rays near the terminator onto the smooth surface (Hanika 2021). In a white-furnace test, a 12-segment mirror sphere went from 3.3% dark pixels to none, a strongly normal-mapped mirror plane from 0.46 to 0.98 of the dome, and terminator steps from 2.1% to 0.45%, at no measurable cost in render time.
MaterialX BSDF graphs. The teapot and lion use standalone BSDF nodes (conductor, dielectric, Oren–Nayar, sheen) combined by layer and mix. The renderer now flattens these graphs onto its layered BSDF. Value graphs that feed them (multiply, power, ln and exp, normal maps and so on) become texture expressions: constants fold, single images stay images, and everything else is baked per UDIM tile when textures load. A layer map carries coat, sheen and specular weights that vary across the surface.
A mesh bug found along the way. The lion's wooden base stayed cream even with its raw texture bound. The cause was in mesh loading: for meshes made only of triangles, Hydra reports that triangulation left face-varying data unchanged, and that answer was treated as a failure, so such meshes lost their UVs. Pre-triangulated exports are common, so this affected far more than one asset.
Matching the official lighting. Tone curves turned out to matter little. A guessed bright studio scored worse than the built-in sky because its white floor whitened the glaze. Instead, a six-parameter studio (key light direction, size and strength, floor albedo, dome warmth) was fitted to both official images at once: about 200 renders in one process, about 3.5 minutes. Turing Sim's Compare Renders gained a matching studio override.
Gold that leaned yellow. An isolated sphere showed the conductor itself was right. The problem was mixing a metal and a diffuse base under a soft mask: one blended colour paled the gold, and specular weight scaled the metal a second time. Each lobe now averages its parameters over its own share, and the metal keeps its own colour.

New headless tests cover each fix, and each was checked to fail without its fix. The renderer moved to its own TuringRender repository, Turing Sim installs it in editable mode, and all branches were merged after full test runs (TuringRender 45 tests, Turing Sim 520 passed after the final merge).

Result

Against the official renders, measured on object pixels as mean sRGB difference out of 255 at the best exposure: the teapot went from 44.5 (grey fallback) to 29.4 under the built-in sky and 24.1 under the fitted studio; the lion from 51.5 to 40.1 and 20.2.

Limits remain. The glaze's view-dependent absorption is evaluated head-on, so the teapot's blue is lighter and the lion's gold less orange than the official images. Coat and base share one normal. Diffuse surfaces still lose some energy under strong normal maps. Implicit USD spheres are still tessellated coarsely. The next step is view-dependent absorption, which should deepen both the blue glaze and the gold.

背景

Turing RT 是机器人仿真器 Turing Sim 背后的 Metal 路径追踪渲染器，已经有 OpenPBR 风格的分层材质、HDRI 重要性采样和 AI 降噪。今天的问题很直接：为什么渲染出来还是不好看？

第一个结论落在场景内容上，不在渲染器。机器人工作站场景里没有灯，也没有材质，只有显示颜色，所以路径追踪和实时光追渲出来几乎一样：既没有可以反弹的间接光，也没有可以反射的光滑表面。在同一个机器人场景里加上地面、两盏面光和几种真实材质，软阴影和光泽反射马上就出来了。于是工作转向渲染器本身真正出错的地方。

目标与难点

让玻璃和金属在真实资产上站得住，先从一个材质测试场景开始；
支持 ASWF 数字制作示例库里 NVIDIA 的 MaterialX Teapot 和 Lion，它们原来完全显示为灰色；
解释它们和官方渲染图的色差，并把能测量的差距尽量缩小；
让 Turing Sim 始终用上最新的渲染器。

全程的约束是项目规则：每一处渲染改动都要附带与参考图的测量对比。

解决思路与行动

低模平滑网格的黑边。 金属球边缘出现黑圈。换成 256×128 的细网格后黑圈消失，说明问题出在粗面片上插值出来的法线：轮廓被误判成背面，反射射线钻进了表面。修复包括三点：用几何面判断正反面；保证不透明表面的着色法线有效（Cycles 的做法，二次方程的选根用 20 万个随机样本做了数值验证）；明暗交界线附近的阴影射线改从平滑曲面上发出（Hanika 2021）。白炉测试中，12 段镜面球的偏暗像素从 3.3% 降到 0，强法线贴图的镜面平面从环境亮度的 0.46 倍升到 0.98 倍，交界线锯齿从 2.1% 降到 0.45%，渲染耗时没有可测量的增加。
MaterialX BSDF 图。 Teapot 和 Lion 的材质由独立的 BSDF 节点（conductor、dielectric、Oren–Nayar、sheen）通过 layer 和 mix 组合而成。渲染器现在把整张图压平到自己的分层 BSDF 上。驱动它们的数值节点链（multiply、power、ln/exp、法线贴图等）变成纹理表达式：常数直接折叠，单张图仍按贴图读取，其余的在加载贴图时按 UDIM 瓦片烘焙。另外用一张 layer 贴图承载随位置变化的清漆、绒毛和高光权重。
顺带发现的网格 bug。 即使直接绑上原始贴图，Lion 的木底座仍是米白色。原因在网格加载：对于全是三角形的网格，Hydra 会告诉调用方 faceVarying 数据"无需改变"，而这个结果被当成了失败，于是这类网格丢掉了 UV。预先三角化的导出很常见，影响面远不止这一个资产。
匹配官方灯光。 实测发现色调曲线影响很小。手搭的亮白影棚反而比内置天空更差，因为白地面把釉面映白了。最后用 6 个参数的影棚（主光方向、大小和强度，地面反照率，环境光冷暖）同时拟合两张官方图：在一个进程里渲约 200 次，用时约 3.5 分钟。Turing Sim 的 Compare Renders 也加上了对应的影棚覆盖项。
偏黄的金色。 单独渲一个金属球，证明导体本身是对的。问题出在软遮罩下混合金属和漫反射：混成一个颜色会把金色冲淡，高光权重还会把金属再缩放一次。现在每个波瓣只在自己所占的份额里平均参数，金属保留自己的颜色。

每处修复都有新的无窗口测试，并确认去掉修复后测试会失败。渲染器搬进了独立的 TuringRender 仓库，Turing Sim 以 editable 方式安装它；所有分支在完整测试后合并（TuringRender 45 个测试；最后一次合并后 Turing Sim 520 个通过）。

结果与下一步

与官方渲染图对比（只看物体像素，在最佳曝光下计算 sRGB 平均差，满分 255）：Teapot 从 44.5（灰色回退）降到内置天空下的 29.4、拟合影棚下的 24.1；Lion 从 51.5 降到 40.1 和 20.2。

仍有局限：釉的视角相关吸收按正对视角计算，所以 Teapot 的蓝比官方浅，Lion 的金不如官方橙；清漆层和底层共用一个法线；强法线贴图下漫反射仍会损失一些能量；USD 隐式球体的细分仍然偏粗。下一步是视角相关吸收，它会让蓝釉和金色都更深。

Turing Sim: Contact Forces You Can Actually Read

alphanine — Sat, 17 Oct 2026 02:49:50 +0000

Each touching pair now gets one named arrow: who pushes whom, and how hard. 每对接触的物体现在只有一个带名字的箭头：谁推谁、推多大。

Before · 改前

After · 改后

Before: one unlabelled peak reading. After: the two pads each pressing 21.2 N on the cube.

Before · 改前

After · 改后

The panel lists named pairs and says how to see every push on a body.

Situation

Turing Sim can draw contact forces while physics runs. Before this change every contact point got its own arrow, and the only label was the strongest force. Holding a cube with a two-finger gripper produced a yellow cluster with "2.78 N" next to it. You could not tell which finger pushed, how hard each side squeezed, or whether a robot's own links touching each other were drowning out the contact you cared about.

Task

Make the overlay answer the questions people actually ask when a grasp misbehaves: who is pushing on this body, in which direction, and how hard? It also had to stay readable on a hand with many links, and work the same in the Storm and the native Metal viewports.

Action

One arrow per touching pair. Contact points are grouped by the two bodies involved. Each pair draws one net-push arrow ending at the force-weighted centre of its points; small dots still mark the individual points.
The selection decides what is shown. With nothing selected, the strongest pairs are labelled "left pad → Cube · 14.5 N". Select a body and every push on it is drawn, each oriented onto that body.
Internal contacts are separated. Contacts between parts of one articulation (two fingers of one hand, the pieces of a puzzle cube) are hidden unless "Include self-contacts" is on, so they never crowd out the rest. MuJoCo and Genesis both report which tree each body belongs to.
Details on hover, and glows. Hovering an arrow opens a card with the pressing and friction components. The native Metal view now forwards pointer motion so this works in every viewport mode. An optional glow marks each point, cool to hot with its force.

The commit was tested in isolation: 88 tests passed.

Result

The captures show the same grasp before and after. The old view had one unlabelled reading; the new one shows the right and left pads each pressing 21.2 N on the cube and a follower link touching it at 2.42 N. The Robot panel lists the same pairs and points to selection for the full picture.

This is a debugging aid, not a measurement tool: the numbers are whatever the engine reports, and the two engines still differ.

背景

Turing Sim 在物理运行时可以显示接触力。改动前每个接触点各有一个箭头，唯一的标签是最大的那个力。用两指夹爪夹住方块时，只能看到一团黄色和旁边的 "2.78 N"。看不出是哪根手指在推、两边各夹多紧，也分不清机器人自身连杆之间的接触是不是把真正关心的接触淹没了。

目标与难点

让叠加层回答抓取出问题时最常问的几个问题：谁在推这个物体、往哪个方向、推多大。同时在多连杆的手上也要看得清，并且在 Storm 和原生 Metal 视口里表现一致。

解决思路与行动

每对接触物体一个箭头。 接触点按涉及的两个物体分组。每对画一个合力箭头，终点在按力加权的接触点中心，小圆点仍标出各个接触点。
由选中对象决定显示什么。 什么都不选时，标出最强的几对，比如 "left pad → Cube · 14.5 N"。选中一个物体，就显示所有作用在它上面的推力，方向都朝向这个物体。
内部接触单独处理。 同一个关节体内部的接触（同一只手的两根手指、魔方的块）默认隐藏，勾选 "Include self-contacts" 才显示，不会挤掉其它接触。MuJoCo 和 Genesis 都会报告每个物体属于哪棵运动树。
悬停查看细节，可选发光。 鼠标悬停在箭头上会弹出卡片，显示法向力和摩擦力分量。原生 Metal 视口现在会转发鼠标移动，所以每种视口模式下都能用。可选的发光效果按力的大小由冷到热标出每个接触点。

这个 commit 单独测试：88 项通过。

结果与下一步

截图对比的是同一次抓取。旧视图只有一个没有标注的读数；新视图显示左右夹爪各以 21.2 N 压住方块，另一个随动连杆以 2.42 N 接触。Robot 面板列出同样的接触对，并提示选中物体可以看到全部推力。

这是调试辅助，不是测量工具：数值就是引擎报告的结果，两个引擎之间仍有差异。

Turing Sim: Seeing Sensors in the Viewport and the Robot Panel

alphanine — Sat, 17 Oct 2026 02:43:05 +0000

Sensors now show where they are in the viewport, and the Robot panel shows what they measure. 视口里能看到传感器在哪里，Robot 面板里能看到它们测到了什么。

Before · 改前

After · 改后

After: camera frustums, the LiDAR head and the IMU axes are drawn where the sensors are.

Before · 改前

After · 改后

After: live sensor rows, LiDAR colouring with a legend, IMU charts and the camera preview.

The scan with a distance ramp and its range in the legend.

Situation

With IMUs, LiDARs and cameras simulated, the editor had to show them. The first version listed sensors as plain text and drew LiDAR points in eight shades of one colour. You could not see where a camera pointed until you pressed Play. During the session the user asked for better visualization.

Task

Show each sensor where it is, whether it is live, and what it measures, in every viewport mode and in a panel that fits a narrow inspector.

Action

Sensors in the viewport. Camera frustums with a tab marking image up, LiDAR heads with their field of view, and IMU axes in the viewport's X, Y, Z colours. While simulating, each IMU also draws its measured specific force. These gizmos show while editing too, which makes placing a sensor easy.
A panel that reads at a glance. One row per sensor with its kind, mount and status (Ready, Live, Note). LiDAR scans can be coloured by intensity, distance, height or class with a Turbo legend. IMUs get four seconds of acceleration and angular velocity per axis. Cameras get a live preview of any output.
One recording. Record sensors writes the host's IMU and LiDAR data and the camera worker's frames into one folder.
Fixes found by capturing the real app. The native captures for this post showed three layout problems, each fixed with a regression test. The preset picker had widened the Robot panel from 322 to 402 pt and scrolled it sideways. Stale rows painted "Ready" under "Live". Six camera outputs were elided in a segmented control and now use a full-width picker.

Result

The before/after captures run the same scene. Before, the viewport shows nothing where the sensors are and the panel has no sensor section. After, frustums, the LiDAR head and the IMU axes appear while editing. While playing, the panel shows live rows, the coloured scan with its legend, the IMU charts and the camera. The panel is back to its pre-sensor width.

背景

IMU、LiDAR 和相机都能仿真之后，编辑器得把它们展示出来。第一版用纯文本列出传感器，LiDAR 点只用同一种颜色的八个深浅来画。按下 Play 之前，你看不出相机朝哪儿拍。会话中用户也提出可视化需要更好一些。

目标与难点

在每种视口模式下显示每个传感器的位置、是否在工作、测到了什么；面板要能放进很窄的检查器里。

解决思路与行动

视口里画出传感器。 相机画成视锥，用一个小标记指示画面的上方；LiDAR 画出探头和视场；IMU 画出坐标轴，颜色和视口的 X、Y、Z 轴一致。仿真时每个 IMU 还会画出它测到的比力。编辑时这些也会显示，方便摆放传感器。
一眼能看懂的面板。 每个传感器一行，显示类型、挂载位置和状态（Ready、Live、Note）。LiDAR 扫描可以按强度、距离、高度或类别上色，并带 Turbo 色带图例。IMU 显示最近 4 秒每个轴的加速度和角速度。相机可以实时预览任意一种输出。
一次录制。 Record sensors 把宿主的 IMU、LiDAR 数据和相机 worker 的帧写进同一个文件夹。
在真实 app 里截图时发现并修复的问题。 为这篇文章截原生窗口时发现了三个布局问题，都已修复并加了回归测试。一是预设下拉框把 Robot 面板从 322 撑到 402，导致横向滚动。二是旧行没有及时移除，"Ready" 和 "Live" 两个状态标签叠在一起。三是六个相机输出在分段控件里被省略显示，现在改成全宽下拉框。

结果与下一步

改前改后用的是同一个场景。改前，视口里传感器的位置什么都没有，面板也没有传感器区。改后，编辑时就能看到视锥、LiDAR 探头和 IMU 坐标轴。播放时，面板显示实时状态行、上色的点云和图例、IMU 曲线以及相机画面。面板也恢复到了加传感器之前的宽度。

Turing Sim: Making the USD Schemas Inspector Readable

alphanine — Tue, 29 Sep 2026 04:31:32 +0000

The running editor shows separate USD schema cards, UR10 articulation settings and the source asset window. 运行中的编辑器展示分开的 USD schema 卡片、UR10 关节体设置及源资产窗口。

Each USD schema on the selected cube (Collision, Mesh Collision, Rigid Body, Mass) now sits in its own card with its USD schema name.

The selected UR10 shows Translate, Rotate, Scale and Articulation Root settings in the running editor. / 运行中的编辑器展示 UR10 的平移、旋转、缩放及 Articulation Root 设置。

The source asset window shows a hierarchy, preview and Properties/Raw USD tabs. / 源资产窗口展示层级、预览及 Properties/Raw USD 标签。

Situation

Turing Sim is an experimental OpenUSD editor and robotics simulation workbench. Its Schemas tab lists every schema on the selected prim, including physics APIs, and every edit goes through the document's undo history. On September 28 the tab worked, but it was hard to read. A viewport tool section sat between the name and the Transform, schemas without properties printed "No additional properties", and the Rigid Body and Mass fields ran together with no visible boundary. Transform showed only the transform parts already stored in the file, so the UR10 showed Position but no Rotation or Scale.

Task

The goal was to make the tab read the way USD is structured, following the conventions of Isaac Sim's Property panel. Every property should clearly belong to one schema, Transform should own translate, rotation and scale, and the Articulation Root should expose settings such as self collision. The difficulty was doing this without hardcoding robot or schema names, which the project rules forbid, and without breaking undo.

A second problem appeared during testing. After moving the UR10 with the gizmo, the Transform section switched to "custom transform stack" and the position fields went stale.

Action

One card per schema. Each schema is now a bordered card. Its header shows a readable name on the left, the USD schema identifier (for example PhysicsRigidBodyAPI) on the right, and a remove button for applied APIs. Removing a schema also clears its values in the current edit layer, as one undo step. Cards that are mostly arrays, such as mesh topology, start collapsed.
Isaac Sim-style rows. Rows are two columns, label and value, and stack when the panel is narrow. Labels come from the schema's own displayName metadata, such as "Center of Mass". A dot at the end of each row shows whether the value is authored: hollow for the schema default, blue when set in the current edit layer (click to revert), grey when set in another layer. Booleans are checkboxes, and physics fields whose USD default is infinity read as "Auto".
Transform owns position, rotation and scale. Translate, Rotate (or Orient) and Scale always appear, named after their USD transform ops. Rotation switches between Euler degrees and a quaternion. Either input is converted to whichever form the prim already stores, so Isaac-style orient quaternions stay quaternions.
The gizmo edits the real transform. The cause of the stale fields was that drags wrote a separate matrix op instead of changing the prim's own translate and rotate values. Drags now update those values directly. The matrix op remains only as a fallback for transforms that position/rotation/scale cannot represent. Older matrix ops are merged back on the next edit.
Articulation Root settings. The card counts the rigid bodies and joints found under the root and offers the PhysX articulation settings: enabled, self collisions, solver iterations, and sleep and stabilization thresholds. Editing one writes the value and applies the PhysX articulation schema in the same undo step.
Source asset editor. The separate editor for referenced assets now uses the same glass cards and segmented tabs as the main window. A user-provided screenshot of the native editor shows the revised layout; editing behavior in that window has not been checked in a headed session.

Regression tests cover each new edit path with undo and redo, plus a check of all six Euler axis orders against USD's own rotation ops. The full offscreen suite gave 525 passed, 3 failed and 6 skipped. The three failures were older tests still looking up the field by its previous name, "Position X". After updating them, the affected test files pass (61 tests). The whole suite has not been re-run since that fix. The three screenshots at the top show the running app after the changes: separate schema cards, the UR10 Transform and Articulation Root panels, and the source asset editor. They show visible layout and values, not an end-to-end edit or save. The other checks were headless, and today's changes are not yet committed.

Result

In the running editor, a cube with collision, rigid body and mass now shows four clearly separated cards, and each field sits under the schema that owns it. Gizmo moves now show up in the Transform fields, and the UR10's articulation settings can be edited in place.

Next steps are to commit and push these changes, re-run the full suite, test the gizmo against real robot assets in the native viewport, and check the Translate fields at narrow panel widths.

背景

Turing Sim 是一个实验性的 OpenUSD 编辑器和机器人仿真工作台。Schemas 页会列出所选 prim 上的所有 schema（包括物理 API），所有修改都进入文档的撤销历史。9 月 28 日时，这个页面能用，但很难读：视口工具区夹在名称和 Transform 之间；没有属性的 schema 显示“No additional properties”；刚体（Rigid Body）和质量（Mass）的字段之间没有边界，混在一起。Transform 只显示文件里已写入的部分，所以 UR10 只有位置，没有旋转和缩放。

目标与难点

目标是让这个页面按 USD 的结构来呈现，并参考 Isaac Sim 属性面板的做法：每个属性都明确属于一个 schema；Transform 负责位置、旋转和缩放；Articulation Root 能设置自碰撞等参数。难点在于不能硬编码机器人或 schema 名称（项目规则禁止），也不能破坏撤销。

测试时又发现一个问题：用 gizmo 移动 UR10 后，Transform 变成“custom transform stack”，位置数值不再更新。

解决思路与行动

每个 schema 一张卡片。 每个 schema 现在是一张带边框的卡片。标题栏左边是易读的名称，右边是 USD schema 标识（例如 PhysicsRigidBodyAPI），已应用的 API 还有删除按钮。删除 schema 时会一并清除它在当前编辑层的数值，整体是一步撤销。以数组为主的卡片（如网格拓扑）默认折叠。
Isaac Sim 风格的属性行。 每行左边是标签、右边是数值，面板较窄时自动上下排列。标签来自 schema 自带的 displayName 元数据，例如“Center of Mass”。行尾的圆点表示数值是否已写入：空心是 schema 默认值；蓝色表示已写入当前编辑层（点击可恢复默认）；灰色表示写在其他层。布尔值改为复选框；USD 默认值为无穷大的物理字段显示为“Auto”。
Transform 负责位置、旋转和缩放。 Translate、Rotate（或 Orient）和 Scale 始终显示，名称与 USD 变换操作一致。旋转可在欧拉角和四元数之间切换，任一种输入都会转换成 prim 原本使用的形式，因此 Isaac 风格的 orient 四元数仍保持为四元数。
gizmo 直接修改真实变换。 数值不更新的原因是：拖动时写入的是一个额外的矩阵操作，而不是修改 prim 自身的平移和旋转值。现在拖动直接更新这些值。只有位置/旋转/缩放无法表示的变换才保留矩阵操作作为后备；旧的矩阵操作会在下次编辑时合并回去。
Articulation Root 设置。 卡片会统计根节点下找到的刚体和关节数量，并提供 PhysX 关节体设置：启用、自碰撞、求解器迭代次数、休眠阈值和稳定阈值。修改其中一项时，会在同一步撤销里写入数值并应用 PhysX 关节体 schema。
源资产编辑器。 引用资产的独立编辑器改用与主窗口相同的玻璃卡片和分段标签。用户提供的原生编辑器截图展示了新版布局；该窗口中的编辑操作尚未经过有界面验证。

每条新的编辑路径都有带撤销/重做的回归测试，另外还用 USD 自身的旋转操作验证了全部六种欧拉轴顺序。完整的离屏测试结果为 525 项通过、3 项失败、6 项跳过。3 项失败都是旧测试仍按原名称“Position X”查找字段；更新后，相关测试文件（61 项）全部通过，但此后尚未重新运行整套测试。文章顶部的三张截图均展示改动后运行中的应用：分开的 schema 卡片、UR10 的 Transform 与 Articulation Root 面板，以及源资产编辑器。它们证明的是可见布局与数值，并不能证明完整的编辑或保存流程。其余检查均为无界面运行，今天的改动也还没有提交。

结果与下一步

在运行中的编辑器里，一个带碰撞、刚体和质量的立方体现在显示为四张清晰分开的卡片，每个字段都在它所属的 schema 下面。gizmo 的移动会直接反映在 Transform 字段里，UR10 的关节体设置也可以直接编辑。

下一步：提交并推送这些改动，重新运行整套测试，在原生视口中用真实机器人资产测试 gizmo，并检查窄面板下的 Translate 字段。

From Localhost to a Built-In Tab: Keeping Story Engine Beside the Conversation

alphanine — Mon, 28 Sep 2026 05:31:04 +0000

Situation

Story Engine began with a review page opened through a local website. It provided a place to browse drafts, read an article and edit it before publication. The project conversation lived elsewhere.

Embedding the entire review page directly in the conversation introduced another layout problem. A long article could occupy most of the chat, making it awkward to move between discussing a change and reviewing the draft.

Task

The goal was to give Story Engine its own review Tab beside the conversation, while preserving the existing interface and local drafts. The chat needed a small entrance; the article needed enough room to read.

This also required a clear boundary between the application and its host. Story Engine could request a presentation mode, while the client controlled where and how that workspace appeared.

Action

The implementation packages the existing page, styles, scripts and brand assets as an MCP UI resource. In the hosted interface, actions go through the same business service used by the browser version. This MCP route serves the interface without starting a localhost HTTP listener and continues using the local data store.

On initialization, the entrance requests the host's expanded presentation once. In this client, that presentation provides the built-in Tab. The inline conversation keeps a compact launcher, with an “Open Tab” button available if the automatic request does not switch the view. Closing the Tab does not start an automatic reopening loop.

The move also exposed a resource-version problem. A newer page could otherwise reach an older process whose registered tools had not changed. Each new server process now retains its bundled UI snapshot alongside its tool registrations. Loading newly added tools still requires the client to reload the updated server.

Result and next steps

The accompanying user-provided screenshot shows Story Engine's draft library in a named Tab on the right, with the project conversation still visible on the left. It provides direct evidence of the opened workspace in this client. The opening transition itself was not recorded, so the image does not establish whether that particular Tab opened automatically.

Source inspection and simulated-host tests cover the compact entrance, the presentation request, fallback behavior and shared operations. The screenshot complements those checks by showing the actual host layout. It does not establish that every review action has been exercised inside that Tab.

The practical improvement is that the conversation and draft workspace can remain visible together. Story Engine still runs locally on the user's Mac. The next step is to verify the complete opening, review and return journey across supported clients and a clean installation, including the behavior of older running plugin processes.

从 localhost 到内置 Tab：让 Story Engine 审稿区留在对话旁

背景

Story Engine 最初通过本地网站打开审稿页，用户可以在里面浏览草稿、阅读文章，并在发布前编辑。项目讨论则留在另一处对话中。

后来把完整审稿页嵌进对话，又产生了布局问题：长文章会占据大部分聊天空间，在讨论修改和阅读草稿之间切换变得不够顺手。

目标与难点

目标是让 Story Engine 在对话旁拥有独立的审稿 Tab，同时保留已有界面和本地草稿。对话只需要一个紧凑入口，文章则需要足够的阅读空间。

应用与宿主之间的职责也需要明确：Story Engine 可以请求一种呈现方式，工作区最终放在哪里、怎样出现，由客户端决定。

解决思路与行动

当前实现把原有页面、样式、脚本和品牌素材打包为 MCP UI 资源。在宿主内打开时，界面操作通过与浏览器版相同的业务服务处理。这条 MCP 路径不需要启动 localhost HTTP 监听，仍然使用原来的本地数据存储。

入口初始化后，会请求一次宿主的展开呈现方式；在这个客户端中，它对应内置 Tab。对话内保留紧凑的启动入口，如果自动请求没有切换视图，用户仍可点击“打开 Tab”。关闭 Tab 后，也不会进入反复自动打开的循环。

这次迁移还暴露了资源版本问题：新页面可能连接到仍注册着旧工具的进程。现在，每个新启动的服务进程都会固定保存与自身工具注册配套的界面快照。新增工具要真正进入当前客户端，仍需重新加载更新后的服务。

结果与下一步

附图由用户提供，真实显示了右侧名为 Story Engine 的 Tab 和草稿列表，左侧项目对话仍然可见。这直接证明了该客户端中已经打开的工作区。打开过程没有录制，因此这张静态截图不能确定当时是否自动切换到了 Tab。

源码核对和模拟宿主测试覆盖了紧凑入口、呈现请求、回退行为，以及共用的业务操作。真实截图补充了实际宿主布局的证据，但不能据此认定所有审稿操作都已在这个 Tab 中逐项验证。

实际变化是：项目讨论和草稿工作区可以同时留在视野内。Story Engine 仍在用户的 Mac 上本地运行。下一步需要在支持的客户端及干净安装环境中，验证打开、审稿和返回对话的完整路径，并继续检查旧插件进程的兼容行为。

Turing Sim: Turning a UI Refresh into a Shareable Milestone

alphanine — Mon, 28 Sep 2026 05:15:31 +0000

Turing Sim: Turning a UI Refresh into a Shareable Milestone

Situation

Turing Sim is an experimental OpenUSD editor and robotics simulation workbench. Its interface needs to keep the 3D viewport central while giving scene editing, inspection, and simulation controls a coherent place to live. The project also has a strict rule for interactive USD edits: they must enter the document command history so users can undo and redo them.

The project owner reported a much better interface after work with Claude Code. On September 27, the immediate need was to capture that work in a reviewable Git commit and make it available on GitHub. The commit includes more than visual changes, so this is a record of the milestone rather than a claim that every feature was built that day.

Task

The goal was to preserve the current application work without sweeping generated output into source control, check what could be verified headlessly, and synchronize it with the published repository. A second challenge was communicating the limits of that verification: the local Python environment does not include every binding used by the full desktop editor.

Action

The recorded changes establish a shared Qt design system with appearance modes, reusable controls, icons, and glass surfaces. The accompanying screenshot shows the newer glass treatment across the window chrome, dock tabs, panels, and viewport tools. The UI work also spans dock tabs, workspace layout, editor chrome, viewport controls, and robot panels. Separately, the commit adds a common runtime contract for Genesis and MuJoCo, a MuJoCo conversion path, and physics authoring through the undoable document command flow. These are distinct areas of work collected in one milestone commit.

The full headless test command completed with 372 passing tests and 14 skips. It also reported 12 failures and 3 errors. Several editor tests could not import pxr.Usdviewq; native schema subprocess tests crashed in the local macOS/OpenUSD environment. Those outcomes prevent a clean-suite claim and call for validation in the intended runtime. The generated MuJoCo warning log was left out of the commit.

GitHub authentication was refreshed. The remote repository had separate commit history but the same pre-change file tree, so the new commit was replayed onto the remote main and pushed without replacing its existing history. The resulting commit is 5881d6f.

Before and after: the interface

Earlier interface: flat charcoal panels and underlined tabs.

Current interface: glass surfaces and grouped controls around the viewport.

The earliest UI capture available for this comparison shows a flatter, mostly charcoal workspace: underlined Objects/Layers tabs, a fixed Inspector, and a small floating tool strip. The current capture shows blue-tinted glass around the viewport, capsule-shaped dock tabs and toolbar groups, a searchable scene tree, and more prominent floating viewport controls. The 3D canvas remains the visual center in both.

These screenshots use different selection and camera states. They illustrate changes in layout and visual language, not a controlled rendering-quality or performance comparison.

Result

The UI and simulation work is now recorded and available for review on GitHub. The design system documentation gives future panels a consistent visual vocabulary, while the engine adapter documentation lays out a path for comparable runtime checks. It explicitly treats benchmark numbers as unmeasured until the benchmark scripts are run and recorded.

The next validation step is to run the editor and schema checks in the supported macOS launch environment, investigate the observed failures, and record measured engine results before drawing performance conclusions.

Turing Sim：将界面更新整理成可分享的里程碑

背景

Turing Sim 是一个实验性的 OpenUSD 编辑器和机器人仿真工作台。界面需要让三维视口保持中心地位，同时清晰地组织场景编辑、属性检查和仿真控制。项目还有一条重要约束：交互式 USD 修改必须进入文档命令历史，才能可靠地撤销和重做。

项目负责人反馈，经过与 Claude Code 协作，界面已有明显改善。9 月 27 日的直接目标是把当前工作保存为可审查的 Git 提交，并同步到 GitHub。本次提交不仅包含视觉更新；提交日期并不能证明每项功能都是当天实现的。

目标与难点

这次需要完整记录应用代码，避免把生成的运行日志混入版本库，完成可在无界面环境运行的验证，并与已发布的仓库同步。另一个难点是准确说明验证边界：本地 Python 环境缺少桌面编辑器所需的部分绑定。

解决思路与行动

提交中的界面工作建立了共享的 Qt 设计系统，包括外观模式、可复用控件、图标和玻璃质感表面。配图展示了新版玻璃效果在窗口工具栏、停靠标签、面板和视口工具上的应用；还覆盖了停靠标签、工作区布局、编辑器工具栏、视口控件和机器人面板。另一部分工作加入了 Genesis 与 MuJoCo 的统一运行接口、MuJoCo 转换路径，以及通过可撤销文档命令进行的物理属性编辑。这些不同方向的改动被整理进同一个里程碑提交。

完整的无界面测试运行结果为 372 项通过、14 项跳过，同时有 12 项失败和 3 项错误。部分编辑器测试无法导入 pxr.Usdviewq；原生 schema 子进程测试在本地 macOS/OpenUSD 环境中崩溃。因此目前不能声称整套测试通过，仍需在目标运行环境继续验证。生成的 MuJoCo 警告日志没有纳入提交。

刷新 GitHub 登录后，发现远端仓库具有独立的提交历史，但变更前的文件树与本地一致。于是将新提交接到远端 main 的历史之后再推送，保留了远端已有历史。最终提交为 5881d6f。

初版与现在：界面变化

早期界面：深灰色平面面板与下划线标签。

当前界面：视口周围采用玻璃质感与分组控件。

目前能找到的最早界面截图以较平面的深灰工作区为主：Objects/Layers 使用下划线标签，Inspector 固定在右侧，视口中只有较小的浮动工具条。新版截图则展示了视口周围的蓝色玻璃质感、胶囊形停靠标签与工具栏分组、可搜索的场景树，以及更醒目的浮动视口控件。两版都把三维画布放在视觉中心。

两张截图的选择状态和相机视角不同，因此这里只比较布局与视觉语言，不据此判断渲染质量或性能。

结果与下一步

这批界面与仿真工作现已记录在 GitHub，供团队审查。设计系统文档为后续面板提供统一的视觉语言；引擎适配文档则定义了比较不同运行后端的方式。文档明确指出，基准测试数值仍需实际运行并记录，当前不能据此得出性能结论。

下一步是在受支持的 macOS 启动环境中验证编辑器和 schema，调查已观察到的失败，并在比较引擎性能前取得真实测量结果。

From Hello World to Project Retrospectives: Bringing Codex Work into a Publishing Workflow

alphanine — Tue, 08 Sep 2026 05:15:09 +0000

English

S · Situation

We are building Story Engine: a product that turns the process of working on a project with AI into an article, making the outcomes easier to share. Our earlier exploration of publishing platforms exposed a problem: successfully verifying an account does not necessarily mean that publishing through its API will work.

Today, our first Hello World post went live on DEV.to. This small milestone lets us move on to the core experience: finishing a project without having to reconstruct the entire story from memory before writing a retrospective.

T · Task and Challenges

The next goal is to say “summarize today's or this week's project progress” in Codex and receive a STAR draft covering progress, challenges, solution approaches, and results.

The main challenge is the scope of the evidence. Code can show what changed, but not necessarily why; the current discussion may not cover the entire week. Another challenge is review consistency: the text the user approves should be exactly the text that gets uploaded.

A · Actions and Approach

We implemented the first version as a project-level Codex skill. It uses the current project's discussions and verifiable materials, defines the reporting period, and distinguishes plans from verified results and unfinished work. STAR provides the narrative structure; missing facts are not filled in with invented details.

Generated articles first enter a local draft store. The web app provides a dedicated “Codex Progress Drafts” view showing the article, source coverage, and supporting notes. Those notes remain local; the publishing request contains only the title and article body.

The publishing flow checks the reviewed draft version and the target account. If the text changes, it must be reviewed again. We also retain the existing duplicate-publication protection: when a request's outcome is uncertain, automatic retries stop so the platform's actual state can be checked first.

R · Results and Next Steps

The user has confirmed that Hello World was successfully published. We have also implemented the progress-drafting instructions, local import, web review, and DEV.to publishing entry point. All 13 automated tests passed, including checks for custom article request formatting and draft-change detection.

These tests use a simulated remote service; they do not establish that this progress article has been published. The next step is to review a real project draft, check its narrative and factual accuracy, and complete a live publication. Coverage across multiple tasks still depends on the project materials Codex can access at the time.

中文

S · 背景

我们在做 Story Engine：把与 AI 一起推进项目的过程整理成文章，让项目成果更容易被分享。此前的发布渠道探索暴露了一个问题：能验证账号，并不意味着就能顺利通过 API 发文。

今天，第一篇 Hello World 已在 DEV.to 发布。这个小里程碑让我们可以继续验证更核心的体验：项目做完后，不再从头回忆如何写一篇复盘。

T · 目标与难点

接下来的目标是，在 Codex 中说一句“总结今天或本周的项目进度”，就得到一份包含进度、难点、解决思路和结果的 STAR 待审稿。

难点在于证据边界。代码能说明改了什么，却不一定能说明为什么改；当前讨论也未必覆盖整个星期。另一个问题是审阅一致性：用户批准的正文，应当就是最终上传的正文。

A · 解决思路与行动

我们将第一版做成项目内的 Codex 技能。它以当前项目的讨论和可核对材料为依据，明确时间范围，并把计划、已验证结果和仍未完成的内容分开。STAR 结构负责组织叙事，缺失的事实不会靠补写来填满。

生成后的文章先进入本地草稿库。网页提供独立的“Codex 进度稿”入口，展示正文、覆盖范围和依据。依据留在本地，发布请求只携带标题与正文。

发布流程会核对审阅时的草稿版本与目标账号；正文变化后需要重新审阅。我们沿用已有的重复发布保护：请求结果不确定时停止自动重试，先核实平台状态。

R · 结果与下一步

Hello World 的真实发布已由用户确认。项目进度稿的生成规范、本地导入、网页审阅与 DEV.to 发布入口也已完成实现，13 项自动测试通过，包括自定义文章请求格式和草稿变化检测。

这些测试使用模拟远端，不代表这篇进度文章已经发布。下一步是审阅一份实际项目稿，检查叙事和事实是否准确，再完成一次真实发布。跨任务记录的完整覆盖仍取决于 Codex 当时可访问的项目资料。

Hello World

alphanine — Tue, 08 Sep 2026 04:55:32 +0000

Hello, world!

This is a test post from Story Engine, a project that turns AI-assisted project discussions into blog posts.

This first post is here to verify the publishing connection. More project stories will follow.