This is a simplified guide to an AI model called Trellis2 maintained by Fishwowater. If you like these kinds of analysis, you should join AImodels.fyi or follow us on Twitter.
Overview
TRELLIS.2 is a 4-billion-parameter 3D generative model built by fishwowater that converts single images into high-fidelity 3D assets with full physically-based rendering (PBR) materials. The model uses a novel "field-free" sparse voxel structure called O-Voxel and a flow-matching transformer architecture to handle complex topologies—including open surfaces, non-manifold geometry, and internal structures—that traditional iso-surface methods cannot represent. The single most important distinction from earlier 3D generation approaches is that TRELLIS.2 generates 3D shape and texture simultaneously in a compact latent space, enabling generation of fully textured assets with base color, roughness, metallic, and opacity maps at resolutions from 512³ to 1536³ voxels, completing in 3 to 60 seconds depending on resolution and GPU hardware.
Best use cases
Product visualization and e-commerce asset generation. Retailers and marketplaces can convert product photographs into interactive 3D models for web storefronts or AR try-on experiences. The model's ability to capture PBR materials means the generated assets render correctly under different lighting conditions, a critical requirement for photorealistic product displays. You control texture resolution up to 8192×8192 pixels and mesh decimation targets up to 2 million faces, allowing you to optimize for web performance or high-fidelity offline rendering.
Character and creature design from concept art. Game developers and animation studios can feed 2D character concept art or illustrations into the model to rapidly prototype 3D character meshes. The O-Voxel representation handles clothing, hair, and appendages without the topological constraints that plague signed distance field (SDF) methods, so loose fabric and separated geometry render correctly. The generated GLB files export with materials ready for further refinement in Blender, Maya, or game engines.
Architectural and environmental asset creation. Environmental artists use TRELLIS.2 to generate building facades, interior furnishings, and landscape elements from photographs. The model correctly interprets depth and perspective from single images, reconstructing geometry that matches the 2D input while inferring occluded regions. The inclusion of material properties enables rapid iteration without manual texture painting.
Rapid prototyping for AR and VR applications. Developers building immersive experiences can convert environmental photographs or product images into 3D-ready assets in seconds, accelerating the content pipeline. The model outputs standard GLB format compatible with Three.js, Babylon.js, and game engines, and includes both geometry and textures in a single export.
Iterative design refinement with texture generation. Beyond image-to-3D, TRELLIS.2 includes shape-conditioned texture generation, allowing you to fix or regenerate surface materials independently. Start with one image, generate a 3D model, then use the shape-conditioned pipeline to apply textures from different reference photographs or written descriptions to the same geometry, enabling rapid style exploration without re-running full generation.
Limitations
Requires clean input images with clear object focus. The model performs best on photographs with distinct subject matter and uncluttered backgrounds. The preprocess_image option removes background and crops automatically, but heavily occluded or ambiguous inputs produce degraded results. Complex multi-object scenes or images with heavy shadows and reflections confuse the model's geometry inference.
Generation time scales steeply with resolution. 512³ resolution completes in roughly 3 seconds, but 1024³ takes 17 seconds and 1536³ takes 60 seconds on NVIDIA H100 hardware. This non-linear cost relationship means high-resolution outputs require substantial compute for production pipelines. No real-time inference capability exists.
Hardware requirements are substantial. The Replicate deployment requires NVIDIA GPUs with at least 24 GB of VRAM; the README specifies testing on A100 and H100 GPUs only. Inference on consumer-grade GPUs (RTX 3090, RTX 4090) may hit memory limits, and older architectures like V100 lack support for the default flash-attention backend. The codebase defaults to CUDA 12.4 and PyTorch 2.6.0, with compatibility constraints on CUDA Toolkit versions.
Generated assets require post-processing for production use. The exported GLB files include alpha channel information but do not enable transparency by default; you must manually connect the alpha channel to material opacity in 3D software. Mesh decimation targets are capped at 2 million faces, and some generated geometry may require cleanup or optimization in external tools depending on downstream requirements. The texture size maximum of 8192×8192 pixels is generous but not unlimited.
Limited by single-image input. The model infers 3D geometry and materials from a single viewpoint, so occluded regions and back-facing surfaces are hallucinated based on learned priors. Multi-view input or sketch-guided conditioning is not supported, limiting precision for objects with critical hidden details. The model's understanding of object size and scale depends entirely on visual context, potentially producing geometrically inconsistent results for ambiguous inputs.
Opacity and transparency handling is imperfect. While the model generates opacity maps, the conversion from O-Voxel representation to GLB may not preserve all transparency nuances. Complex semi-transparent materials like frosted glass or translucent fabrics may not render as expected without additional refinement.
No explicit safety or content filtering mentioned. The model accepts any image URL, and the README does not document content policies or refusal behavior for problematic inputs.
How it compares
TRELLIS.2 versus TRELLIS.2-4B (Hugging Face): The Replicate deployment of TRELLIS.2 and the Hugging Face model checkpoint use the same underlying 4B-parameter weights, so output quality and speed are identical. The Replicate version handles image preprocessing, GLB export, and video rendering as managed services, reducing setup complexity; the Hugging Face checkpoint requires you to install dependencies (CUDA 12.4, flash-attention, nvdiffrast, cumesh, o-voxel), manage VRAM allocation, and write export code yourself. Choose Replicate for quick prototyping or production inference without infrastructure; choose Hugging Face if you need to fine-tune the model, integrate it into a local pipeline, or avoid cloud dependencies.
TRELLIS.2 versus trellis (by fal-ai): Both models perform image-to-3D generation, but the original TRELLIS (prior to the .2 iteration) uses a different latent representation and may have lower fidelity or slower inference. TRELLIS.2 explicitly improves topology handling for complex shapes and adds multi-channel material generation (roughness, metallic, opacity). If you require the absolute latest results and can accept slightly longer inference times, TRELLIS.2 is superior; if you need faster generation and the original TRELLIS quality is sufficient, the alternative may cost less per inference.
TRELLIS.2 versus trellis (by firtoz on Replicate): This appears to be an alternative deployment of an earlier TRELLIS version. TRELLIS.2 is the newer, more capable model with superior material handling and topology support. Choose TRELLIS.2 unless you have a specific reason to use the earlier version (e.g., lower cost, compatibility with existing workflows).
TRELLIS.2 versus trellis-2 (by fal-ai): Both are the latest TRELLIS.2 model running on different inference platforms (Replicate versus fal-ai). Replicate and fal-ai differ in pricing, rate limiting, and API design; they likely use identical model weights and produce nearly identical outputs. Choose based on which platform better integrates with your existing architecture and pricing model.
TRELLIS.2 versus TRELLIS-image-large (by microsoft): TRELLIS-image-large is the predecessor model from the original TRELLIS paper (arxiv 2412.01506). It uses the same O-Voxel representation but is trained on different data and may produce lower-quality geometry or materials. TRELLIS.2 explicitly improves upon this model with a larger parameter count (4B vs. unspecified), better material generation, and refined training. Always prefer TRELLIS.2 for new projects unless reproducing results from the original paper.
Technical specifications
Architecture: Flow-matching transformer with 4 billion parameters, operating on compact structured latents (O-Voxel representation). The model uses a sparse 3D VAE with 16× spatial downsampling to encode 3D assets, enabling generation in a compact latent space. Generation proceeds in stages: sparse structure prediction (12 steps default), shape SLat (structured latent) refinement (12 steps default), and texture SLat refinement (12 steps default), with each stage configurable for quality/speed tradeoff.
Resolution and output: Generates 3D assets at 512³, 1024³, or 1536³ voxel resolution. Texture resolution for GLB export ranges from 1024 to 8192 pixels (default 4096). Mesh decimation targets range from 100,000 to 2,000,000 faces (default 1,000,000). Exported GLB files include geometry (vertices, faces) and four material channels: base color, roughness, metallic, and opacity. Video preview renders at 15 FPS.
Inference speed (NVIDIA H100):
- 512³: ~3 seconds total (2s shape + 1s texture)
- 1024³: ~17 seconds total (10s shape + 7s texture)
- 1536³: ~60 seconds total (35s shape + 25s texture)
Input requirements: Single image in any standard format (JPEG, PNG, WebP, etc.) provided as a URL. Image preprocessing (background removal, cropping) is optional but enabled by default.
Training data: Trained on Objaverse-XL dataset (the README mentions this for training; specific composition of pretraining data not disclosed).
Dependencies (for local deployment): CUDA Toolkit 12.4 (or compatible version), PyTorch 2.6.0, flash-attention backend (or xformers as fallback), nvdiffrast, nvdiffrec, cumesh, o-voxel, flexgemm. Linux-only (tested on Linux; Windows/macOS not officially supported).
Quantization and model format: Weights distributed via Hugging Face as PyTorch checkpoints. No quantized variants or ONNX exports mentioned.
Licensing: MIT License (per GitHub repository).
Model inputs and outputs
Inputs
- image (string, URI format, required): URL to input image for 3D generation.
- seed (integer, default 42): Random seed for generation reproducibility.
- randomize_seed (boolean, default false): Override seed with a random value.
- preprocess_image (boolean, default true): Automatically remove background and crop input image.
- return_no_background (boolean, default false): Return the preprocessed image (background removed) in addition to 3D output.
- generate_video (boolean, default true): Generate rendered video preview of the 3D asset.
- generate_model (boolean, default true): Generate and export GLB 3D model file.
- pipeline_type (enum, default "1024_cascade"): One of ["512", "1024", "1536", "512_cascade", "1024_cascade", "1536_cascade"]. Determines resolution and quality tier.
- sparse_structure_steps (integer, default 12, min 1, max 50): Diffusion steps for sparse structure prediction (higher = better quality, slower).
- sparse_structure_guidance_strength (number, default 7.5, min 0, max 15): Classifier-free guidance strength for structure generation.
- sparse_structure_guidance_rescale (number, default 0.7, min 0, max 1): Guidance rescaling factor for structure.
- sparse_structure_rescale_t (number, default 5, min 1, max 6): Time-step rescaling for structure diffusion.
- shape_slat_steps (integer, default 12, min 1, max 50): Diffusion steps for shape SLat refinement.
- shape_slat_guidance_strength (number, default 7.5, min 0, max 15): Guidance strength for shape refinement.
- shape_slat_guidance_rescale (number, default 0.5, min 0, max 1): Guidance rescaling for shape.
- shape_slat_rescale_t (number, default 3, min 1, max 6): Time-step rescaling for shape diffusion.
- tex_slat_steps (integer, default 12, min 1, max 50): Diffusion steps for texture SLat refinement.
- tex_slat_guidance_strength (number, default 1, min 0, max 15): Guidance strength for texture generation.
- tex_slat_guidance_rescale (number, default 0, min 0, max 1): Guidance rescaling for texture.
- tex_slat_rescale_t (number, default 3, min 1, max 6): Time-step rescaling for texture diffusion.
- texture_size (integer, default 4096, min 1024, max 8192): Output texture resolution (only used if generate_model=true).
- decimation_target (integer, default 1,000,000, min 100,000, max 2,000,000): Target triangle count for mesh decimation (only used if generate_model=true).
Outputs
The output schema references a PredictOutput component but does not detail the exact structure. Based on the README and inputs, outputs include:
-
Generated 3D Model (GLB file): If
generate_model=true, a GLB file containing vertices, faces, and material attributes (base color, roughness, metallic, opacity) at the specified texture resolution and decimation target. -
Rendered Video Preview (MP4): If
generate_video=true, a video file showing the generated 3D asset under various lighting conditions with PBR material preview at 15 FPS. -
Preprocessed Image (optional): If
return_no_background=true, the background-removed and cropped input image.
Getting started
import replicate
# Initialize the Replicate client (requires REPLICATE_API_TOKEN environment variable)
client = replicate.Replicate()
# Run image-to-3D generation
output = client.run(
"fishwowater/trellis2:52e1ad6852599ea10ce8e257635a3c11485cba51c181ea5173e34d9b2955b226",
input={
"image": "https://example.com/your_image.png",
"seed": 42,
"randomize_seed": False,
"preprocess_image": True,
"generate_video": True,
"generate_model": True,
"pipeline_type": "1024_cascade",
"sparse_structure_steps": 12,
"sparse_structure_guidance_strength": 7.5,
"sparse_structure_guidance_rescale": 0.7,
"sparse_structure_rescale_t": 5,
"shape_slat_steps": 12,
"shape_slat_guidance_strength": 7.5,
"shape_slat_guidance_rescale": 0.5,
"shape_slat_rescale_t": 3,
"tex_slat_steps": 12,
"tex_slat_guidance_strength": 1,
"tex_slat_guidance_rescale": 0,
"tex_slat_rescale_t": 3,
"texture_size": 4096,
"decimation_target": 1000000,
},
)
# output contains URLs or file paths to generated GLB and video
print(output)
Frequently asked questions
Q: What image formats and sizes does TRELLIS.2 accept?
A: The model accepts standard image formats (JPEG, PNG, WebP, etc.) via URL. The README does not specify minimum or maximum image dimensions, but single-object, clearly-focused photographs produce the best results. The preprocess_image option automatically crops and centers the object, so you do not need to pre-crop inputs.
Q: Can I control the 3D output resolution independently from texture resolution?
A: Yes, the pipeline_type parameter determines 3D geometry resolution (512³, 1024³, or 1536³), while texture_size independently controls the exported texture resolution (1024 to 8192 pixels). This decoupling allows you to generate high-resolution geometry with lower texture resolution for faster export, or vice versa.
Q: What license applies, and can I use generated assets commercially?
A: TRELLIS.2 is released under the MIT License, which permits commercial use of the model itself. However, commercial use of generated assets depends on your input image's rights and any commercial terms in the Replicate service agreement. Verify that you own or have rights to the input image before commercial deployment of generated 3D models.
Q: How does generation quality vary across different guidance strengths?
A: Higher guidance strengths (e.g., 10+ for structure or shape) push the model to more closely follow the input image, improving fidelity but potentially reducing diversity or introducing artifacts. The defaults (7.5 for structure/shape, 1 for texture) balance quality and coherence. Reducing guidance strength below 5 produces more "creative" or divergent results; exceeding 15 (the maximum) can cause distortion. Rescale parameters (0–1) further modulate the effect; experimenting within the 0.5–0.7 range is recommended.
Q: What happens if I provide an ambiguous or multi-object image?
A: The model's geometry inference depends on visual context to infer scale, material properties, and occlusion. Ambiguous inputs (multiple objects, heavy occlusion, unclear depth cues) often produce incorrect topology or conflated geometry. The optional preprocess_image step automatically crops and centers, but you should manually verify that the input focuses on a single, clearly-defined object for best results.
Q: Is TRELLIS.2 suitable for production rendering pipelines?
A: Yes, with caveats. The model exports standard GLB files compatible with game engines (Unreal, Unity), 3D software (Blender, Maya), and web viewers (Three.js). However, generated geometry may require cleanup in external tools, and transparency (alpha channel) is not enabled by default in the GLB export—you must manually connect material properties in your 3D software. For high-volume asset generation (e.g., e-commerce), the inference cost and time (up to 60 seconds per asset at high resolution) should factor into your pipeline design.
Q: How does TRELLIS.2 handle complex shapes like clothing, hair, or vegetation?
A: The O-Voxel representation was designed to handle open surfaces, non-manifold geometry, and internal structures that traditional signed distance fields cannot represent. Clothing and loose geometry render correctly. However, extremely thin or wispy details (fine hair strands, delicate leaves) may be simplified or merged in the voxel representation, and dense vegetation or complex organic forms can still produce artifacts. Test with your target object class before committing to production use.
Q: Can I regenerate textures on an existing shape without re-running the full pipeline?
A: Yes. The README mentions shape-conditioned texture generation (see example_texturing.py), allowing you to apply new textures to a pre-generated shape. This decoupling saves compute if you want to explore multiple material variations on the same geometry, though the dedicated texture generation pipeline is not exposed via the main Replicate deployment—you would need to use the Hugging Face model locally.
Top comments (0)