DEV Community

Google Developer Experts profile image Tanaike
Tanaike for Google Developer Experts

Posted on

Smart Stowage: Building a 3D Cargo Digital Twin with Gemini 3

#ai #gemini #react #node

fig1

Abstract

This article details the development of Smart Stowage Optimizer, a web-based digital twin for logistics that bridges the gap between physical safety and artificial intelligence. By integrating Gemini 3 Pro, the system solves the 3D Bin Packing Problem (3DBPP) using advanced spatial reasoning. Built with React 19 and Three.js, the application visualizes physics-aware load stability in real-time, offering a comparative analysis between traditional heuristic algorithms and modern generative AI agents.

Introduction

The capabilities of Large Language Models (LLMs) are expanding at an unprecedented rate, moving from text generation to complex reasoning. When Gemini 2.5 was released, I conducted a feasibility study on automating cargo ship stowage planning, published in "Stowage Planning Automation Using Gemini: A Feasibility Study and A Prompt-Based Approach" Ref.

At that time, the implementation was a backend Python script producing static JSON results. While effective as a proof of concept, real-world logistics demands interactive visualization and immediate feedback on physical constraints like the Center of Gravity (COG).

With the arrival of Gemini 3, we can now handle significantly more complex spatial reasoning tasks. This article introduces the Smart Stowage Optimizer, a full-stack Web Application that evolves the previous concept into a live 3D Digital Twin. This application not only calculates optimal packing but also visualizes stability and safety constraints in a high-fidelity 3D environment.

Repository

You can access the full source code here:

https://github.com/tanaikech/Smart-Stowage-Optimizer

System Architecture

To tackle the 3D Bin Packing Problem (3DBPP)—a known NP-hard optimization challenge—I designed a dual-engine architecture. This allows users to compare the speed of deterministic algorithms against the semantic reasoning of modern AI.

  • Frontend: React 19, Tailwind CSS, Lucide React
  • Visualization: React Three Fiber (Three.js), InstancedMesh for performance
  • AI Logic: Google GenAI SDK (Gemini 3 Pro)
  • Algorithms: Custom greedy heuristic implementation

Below is the high-level data flow of the application. You can generate a similar diagram using the prompt below.

[Diagram Suggestion: System Architecture Flowchart]
Prompt for Diagram Generation: "Create a flowchart showing a React Frontend sending cargo data to two branches: one to a Local Heuristic Engine and one to the Gemini API. Both branches merge back into a Validation Layer which calculates Physics (COG), and finally outputs to a 3D Renderer."

fig2

Mermaid Chart Playground

Methodology: Dual-Engine Optimization

1. The Deterministic Approach: Algorithmic Engine

The first engine utilizes a Greedy Shelf-Packing Heuristic, specifically a variation of the First-Fit Decreasing Height (FFDH) algorithm.

  • Logic: It sorts cargo items by height to define "shelves" or layers.
  • Execution: Items are packed sequentially along the X and Z axes, filling one layer before moving to the next.
  • Pros/Cons: This method is computationally inexpensive and ensures a tidy lineup. However, it lacks "common sense"—it cannot understand complex constraints like "do not stack heavy machine parts on top of fragile electronics" unless explicitly hard-coded.

2. The Probabilistic Approach: Gemini 3 Pro Engine

The second engine leverages the Spatial Reasoning capabilities of Gemini 3 Pro. Instead of iterating through a fixed loop, the AI evaluates the entire manifest holistically.

  • Why Gemini 3?: Previous models struggled with 3D coordinate mapping. Gemini 3 Pro shows a marked improvement in maintaining spatial relationships, allowing it to "visualize" the container interior.

  • Workflow: The application prompts the model to act as a Senior Stowage Engineer.

  • Constraints: The prompt enforces non-linear rules:

    • Stability: Keep the Center of Gravity (COG) low.
    • Safety: Heavier items must be placed at y=0.
    • Balance: Lateral and longitudinal distribution for truck axles.

Implementation: Structured JSON Output

To ensure the AI returns usable data, we utilize the responseMimeType: "application/json" configuration in the Google GenAI SDK. This forces the model to adhere to a strict schema, returning a typed array of coordinates rather than conversational text.

// services/geminiService.ts snippet
const response = await ai.models.generateContent({
  model: "gemini-3-pro-preview",
  contents: prompt,
  config: {
    responseMimeType: "application/json",
    responseSchema: {
      type: Type.OBJECT,
      properties: {
        packedItems: {
          type: Type.ARRAY,
          items: {
            type: Type.OBJECT,
            properties: {
              id: { type: Type.STRING },
              x: { type: Type.NUMBER },
              y: { type: Type.NUMBER },
              z: { type: Type.NUMBER },
            },
            required: ["id", "x", "y", "z"],
          },
        },
      },
    },
  },
});
Enter fullscreen mode Exit fullscreen mode

Safety & Physics Validation

AI is probabilistic, but cargo safety must be deterministic. Therefore, the application includes a validation layer:

  1. Boundary Clamping: If the AI suggests a coordinate where an item clips through the wall ($x + width > container_width$), the system automatically clamps the item to the boundary.
  2. COG Calculation: The app calculates the weighted average position of all items ($COG = \frac{\sum (m_i \times p_i)}{\sum m_i}$) and renders a red visual marker in the 3D view. This allows engineers to verify stability instantly.

Application Overview: Smart Stowage Optimizer

Smart Stowage Optimizer acts as a sandbox to simulate loading risks before physical operations begin.

Key Features

  • Digital Twin Visualization: Using react-three-fiber, the scene renders a semi-transparent container with a grid floor. We use InstancedMesh for the cargo items, allowing the app to render hundreds of boxes with a single GPU draw call for high performance.
  • Physics-Aware Metrics: The UI displays real-time volumetric efficiency (%) and total payload weight, changing colors to warn of overloading.
  • Manifest Management: Users can upload CSV files, manually add items, or use industry-standard presets (20ft/40ft ISO containers, EURO Trucks).

Comparative Analysis

Feature Algorithmic (Heuristic) Gemini AI Engine
Speed Instantaneous (<100ms) 3-8 seconds (Inference)
Logic Deterministic (Fixed Rules) Probabilistic (Reasoning)
Stability Basic Height-based layers Advanced COG awareness
Flexibility Rigid packing patterns Adapts to unique constraints
Best For Uniform pallets, high speed Complex manifests, fragile items

Installation and Usage

To run the application locally:

  1. Clone the repository: 
git clone https://github.com/tanaikech/Smart-Stowage-Optimizer
Enter fullscreen mode Exit fullscreen mode
  1. Install dependencies: 
npm install
Enter fullscreen mode Exit fullscreen mode
  1. Configure API Key:

Set your Gemini API key as an environment variable GEMINI_API_KEY in a .env file.

  1. Start the server: 
npm run dev
Enter fullscreen mode Exit fullscreen mode

Workflow

  1. Define Space: Select a preset (e.g., "EURO_TRUCK") or define custom dimensions (W/H/L).
  2. Build Manifest: Upload a CSV or use the "Sample" button to populate test cargo (generators, drums, electronics).
  3. Select Engine: Choose Gemini AI for stability-focused packing or Algorithmic for speed.
  4. Analyze: Rotate the 3D view. Check the red COG marker to ensure it is centered and low.
  5. Export: Download the generated coordinates as a CSV report for the loading crew.

Testing and Demonstration

The following demonstration highlights the difference between the engines. Note how the Algorithmic engine packs tightly but may ignore weight distribution. In contrast, the Gemini engine intelligently places heavier items (blue/green generators) at the bottom and lighter items (orange electronics) on top to lower the Center of Gravity.

The exported CSV data generated by Gemini is structured as follows:

ID,Name,Width,Height,Length,Weight,X,Y,Z
BASE-1,Generator 1,1.2,1,2,2500,0,0,0
BASE-2,Generator 2,1.2,1,2,2500,0,0,2
PAL-1,Heavy Spares,1,0.8,1.2,1200,1.35,0,0
PAL-2,Heavy Spares,1,0.8,1.2,1200,1.35,0,1.2
BOX-A1,Electronics,0.6,0.6,0.8,150,0,1,0
BOX-A2,Electronics,0.6,0.6,0.8,150,0.6,1,0
...
Enter fullscreen mode Exit fullscreen mode

Summary

By integrating Gemini 3 Pro, we have moved beyond simple script-based calculators to a fully interactive Digital Twin. While traditional algorithms are faster for uniform shapes, Gemini demonstrates a unique ability to understand "soft" constraints—such as stability and stackability—that are difficult to program explicitly. This hybrid approach represents the future of logistics software: combining the speed of heuristics with the reasoning of Artificial Intelligence.

  • Hybrid Architecture: Successfully combines deterministic Heuristics with probabilistic AI.
  • Gemini 3 Capability: Confirms that Gemini 3 Pro handles complex spatial reasoning and 3D coordinate mapping significantly better than previous iterations.
  • Visualization Impact: React 19 and Three.js transform abstract data into visual insights, allowing for immediate safety verification.
  • Physics Integration: Real-time Center of Gravity (COG) calculations ensure that AI-generated plans adhere to physical safety standards.

Top comments (0)