Ant Research released 4DAnyone, a system that turns a single uncalibrated video of one person into a full 4D reconstruction, meaning a 3D model that moves through time and can be viewed from angles the camera never occupied. The pipeline first generates multiview-consistent videos of the subject from new viewpoints, then lifts those into 4D Gaussian Splatting. Code is on GitHub and weights are on Hugging Face, and the paper carries an explicit ethics section warning about deepfake misuse and requiring consent from the person filmed.
Key facts
- Input requirements per the repository: 720p or higher, 9:16 portrait, one person, full-body or upper-body framing, at least 121 frames, mild camera motion only.
- Ships inference presets for a 6-view orbit, a 24-view orbit suitable for reconstruction, a 48-view three-pitch-layer setup, and an 8-view frontal arc.
- Posted as arXiv 2608.20335 with a project page, GitHub repository, and Hugging Face model card.
- Named mechanisms: 3D-aware skeleton conditioning, Reference Context Packing, and Target Context Routing.
The hard part of this problem is not generating a plausible image of someone from the side. Image models have done that for years. The hard part is generating a plausible image from the side that agrees with the plausible image from the other side, and with the one from above, frame after frame, so that all of them can be fused into a single consistent object. Inconsistency is invisible in any one picture and fatal to reconstruction. Feed a reconstructor twenty views that each look great but disagree about where the left shoulder is, and you get a smear.
That is what the two named mechanisms are for. Reference Context Packing controls how the original footage is presented to the generator so it stays anchored to the real person, and Target Context Routing controls how information flows to each generated viewpoint so they stay mutually consistent instead of drifting apart. On top of that sits 3D-aware skeleton conditioning, which gives the model an explicit pose scaffold rather than asking it to infer the body's articulation from pixels alone. The analogy is a police sketch artist working from one photo and several witness descriptions: without a shared skeleton to hang details on, each description produces a slightly different face. With one, they converge.
The output feeds Gaussian splatting, which represents a scene as millions of small translucent blobs rather than as a mesh, and which has become the standard way to reconstruct a real scene into something you can move a camera through freely. The repository's 24-view full orbit preset is explicitly labeled as the one suited to reconstruction, which tells you the practical trade: six views for a quick preview, 24 for something you can actually rebuild, 48 across three pitch layers when you need the tops and bottoms of surfaces too.
"Casual" is doing some work in the way this gets described, and the repository is more honest than the summaries. It wants 720p or better, portrait orientation, one person, full-body or upper-body framing, at least 121 frames, and only mild camera motion. That is a well-shot phone video, not a clip grabbed off a security camera. The stated limitations are also specific and believable: the system holds up under occlusion and motion blur, and falls down on loose garments and inaccurate pose estimation, which are exactly the two places where a skeleton scaffold stops helping.
A sibling paper released alongside it, WithEveryone, tackles a different problem in the same family: generating a single coherent image containing five to ten specific named people. It uses identity tokens plus a structured layout planning step and ties each identity to its own annotated region during training, which is the mechanism that stops faces from blending into each other when many are present. That one is not fully released. The project page says the research system was built on a foundation model whose license does not permit checkpoint release, and the team is retraining on an openly releasable base.
The honest caveat is the one both papers put in writing themselves, and it deserves credit rather than the usual eye-roll. 4DAnyone's ethics and impact section names deepfake misuse, identity privacy violation, and copyright infringement directly, and asks that outputs be labeled synthetic and made only with consent. WithEveryone's responsible-use section names impersonation and non-consensual generation and recommends consent plus provenance signaling. Neither of those is enforceable, and 4DAnyone's weights are downloadable today. A system that turns one clip of a person into a controllable 3D version of them is a genuine advance for volumetric capture and a genuine gift to anyone building a convincing fake. Ground Truth has covered content provenance tooling and a tool that strips those marks passing 4,900 stars; the labeling half of the ecosystem is losing ground to the generation half.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)