DEV Community: Aman singh

How I Built a Cinematic 3D Fitness Website with React Three Fiber and Three.js

Aman singh — Sat, 26 Sep 2026 15:18:13 +0000

Building a Cinematic 3D Fitness Website with React Three Fiber and Three.js

Modern fitness websites don't have to look like a simple collection of cards, buttons, and text.

While building my fitness platform ASFORGE FITNESS, I wanted the website to feel more like an interactive experience. The main goal was to combine a fitness platform with a cinematic 3D interface, animated characters, glowing effects, and an AI fitness coach.

In this article, I'll share how I built the 3D experience using React, Vite, React Three Fiber, Three.js, and Drei.

🚀 What I Wanted to Build

The idea was to create a fitness website with:

A cinematic hero section
An interactive 3D character
Character animations
Animated background effects
Stars and particles
Energy rings and glow effects
AI fitness coach integration
Workout and diet sections
Membership plans
A modern responsive interface

Instead of making the 3D scene a completely separate application, I wanted it to work naturally inside a normal React website.

🛠️ Tech Stack

The main technologies I used were:

React
Vite
JavaScript
Three.js
React Three Fiber
Drei
React Three Postprocessing
GLB 3D models
Gemini AI

The normal website UI is built with React, while React Three Fiber is responsible for rendering the 3D experience.

⚛️ Why React Three Fiber?

Three.js is extremely powerful for creating 3D experiences on the web.

However, when working inside a React application, managing Three.js scenes directly can become difficult.

React Three Fiber provides a React-based way to work with Three.js.

Instead of manually creating and managing everything with traditional Three.js code, I could structure the 3D scene using React components.

For example:


jsx
<Canvas>
  <ambientLight />
  <directionalLight position={[2, 4, 5]} />

  <HeroCharacter />

  <Stars />
</Canvas>

This made it much easier to keep the 3D scene organized as part of the React application.

🧍 Adding a 3D Character

One of the most important parts of the hero section was the 3D character.

I used a .glb model and loaded it with Drei's useGLTF.

A simplified version looks like this:

import { useGLTF } from "@react-three/drei";

function HeroCharacter() {
  const { scene } = useGLTF("/models/character.glb");

  return (
    <primitive
      object={scene}
      scale={1}
      position={[0, -1, 0]}
    />
  );
}

The model is then placed inside the React Three Fiber canvas.

The important part was not simply loading the model.

The real challenge was getting the scale, camera position, and character position right.

📷 Camera Position Matters

A 3D model can look completely different depending on the camera.

Initially, my character was either too large, too small, or partially outside the screen.

I experimented with camera settings such as:

<Canvas
  camera={{
    position: [0, 0.5, 6],
    fov: 40,
  }}
>

I then adjusted the character's scale and position until the model stayed inside the hero section while still looking large and cinematic.

This was one of the most important parts of creating the final composition.

🎬 Playing the Character Animation

The 3D model also needed to feel alive.

For animated GLB models, I used Drei's animation support with useAnimations.

A simplified example:

import { useEffect } from "react";
import { useAnimations, useGLTF } from "@react-three/drei";

function AnimatedCharacter() {
  const { scene, animations } = useGLTF(
    "/models/character.glb"
  );

  const { actions } = useAnimations(
    animations,
    scene
  );

  useEffect(() => {
    const animation = Object.values(actions)[0];

    if (animation) {
      animation.reset().fadeIn(0.5).play();
    }

    return () => {
      if (animation) {
        animation.fadeOut(0.5);
      }
    };
  }, [actions]);

  return <primitive object={scene} />;
}

This allowed the model's built-in animation to play inside the website.

✨ Creating the Cinematic Environment

A plain 3D model wasn't enough.

I wanted the scene to feel energetic and connected to the fitness theme.

So I combined several visual effects:

Stars
Particles
Glowing rings
Background effects
Energy effects
Lighting
Bloom
Animated elements

The result was much more dynamic than simply placing a model on a static background.

🌌 Stars and Particles

For the background, I used Drei's Stars component.

For example:

<Stars
  radius={80}
  depth={50}
  count={3000}
  factor={4}
  saturation={0}
  fade
  speed={1}
/>

This gives the background a subtle space-like environment.

I also experimented with additional particle effects to make the scene feel more alive.

💡 Lighting

Lighting makes a huge difference in 3D.

I used multiple light sources instead of relying on only one light.

For example:

<ambientLight intensity={0.5} />

<directionalLight
  position={[3, 5, 2]}
  intensity={2}
/>

The goal was to make the character clearly visible while maintaining a dramatic look.

🌟 Bloom and Glow

To make bright elements feel more cinematic, I used post-processing.

The setup included:

<EffectComposer>
  <Bloom
    intensity={1.2}
    luminanceThreshold={0.2}
    luminanceSmoothing={0.9}
  />
</EffectComposer>

Bloom helped create the glowing appearance around bright objects and energy effects.

This was especially useful for the rings, particles, and other bright elements in the scene.

🔄 Keeping the Scene Animated

A static 3D scene can quickly feel boring.

React Three Fiber provides useFrame, which can be used to update objects on every rendered frame.

For example:

useFrame((state, delta) => {
  mesh.rotation.y += delta * 0.3;
});

This can be used for:

Rotation
Floating effects
Animated rings
Particle movement
Camera effects
Other continuous animations

I used this approach to make different parts of the hero scene feel dynamic.

🧩 Integrating 3D with a Normal React UI

One of the interesting parts of this project was combining a traditional website UI with the 3D scene.

The page still contains normal React elements such as:

Navigation
Buttons
Text
Fitness plans
Workout sections
AI Coach section
Contact forms

The 3D scene is placed inside the hero section rather than replacing the entire website.

Conceptually, the structure looks like:

React Application
│
├── Navigation
│
├── Hero Section
│   ├── Text Content
│   ├── CTA Buttons
│   └── React Three Fiber Scene
│       ├── 3D Character
│       ├── Lights
│       ├── Stars
│       ├── Particles
│       ├── Energy Effects
│       └── Post Processing
│
├── AI Coach
├── Workout Sections
├── Membership
└── Contact

This structure made it possible to use 3D where it actually adds value instead of turning the whole website into a 3D application.

🤖 Connecting the 3D Experience with an AI Fitness Coach

The 3D hero was only one part of the project.

The website also includes an AI fitness coach powered by Gemini.

The AI Coach provides trainer-style interaction while the 3D scene creates the visual identity of the platform.

This combination gives the website two different experiences:

Visual experience:

3D character + animations + cinematic effects

Interactive experience:

AI fitness coach + conversations + fitness guidance

🌍 Multilingual AI Coach

Another part of the project is multilingual support.

The AI Coach can work with a large number of languages, allowing users to interact with the fitness platform beyond a single language.

This became especially useful for the idea of making the platform accessible to users from different backgrounds.

The language handling is kept separate from the 3D rendering system, so the two systems can evolve independently.

🐛 Some Challenges I Faced

Building the 3D section wasn't completely straightforward.

1. Model going outside the screen

The first challenge was positioning the character correctly.

Changing the scale also changed how large the character appeared relative to the camera.

I had to balance:

Camera position
+
Field of view
+
Model scale
+
Model position

until the character looked good on the screen.

2. Animation setup

A GLB model may contain one or multiple animations.

So I had to load the animation clips and start the appropriate action instead of assuming the model would automatically animate.

3. Too many visual effects

Adding more effects does not always make a scene better.

Too many particles, lights, or post-processing effects can make the scene visually noisy and can also increase rendering work.

I therefore treated the effects as supporting elements rather than the main focus.

4. React and Three.js integration

Another challenge was keeping the 3D components modular.

Instead of putting everything into one large component, I separated different responsibilities into components such as:

HeroScene
Character
Energy Effects
Particles
Rings
Lighting

This made the project easier to modify.

📱 Making the Experience Fit the Website

The 3D scene also had to work with the rest of the website.

A cinematic desktop layout can easily break on smaller screens.

So the scene needs to be considered together with:

Hero text
Buttons
Navigation
Screen size
Character scale
Camera position

The goal is not just to make the 3D scene look good on one screen, but to make the overall hero section usable.

🚀 What I Learned

This project taught me that adding 3D to a website is not just about creating a cool model.

The important parts are:

Composition
Camera positioning
Lighting
Animation
Performance
UI integration
Responsive design

React Three Fiber made the process much easier because I could build the 3D experience using a React component structure.

🔮 What's Next?

The ASFORGE FITNESS project is still evolving.

Some of the areas I want to continue improving include:

Better 3D interactions
More fitness-focused animations
Improved mobile performance
More AI features
Better workout personalization
More interactive exercise demonstrations

The long-term goal is to bring the same experience to a mobile application as well.

🏁 Final Thoughts

Building a cinematic 3D website showed me how much difference visual interaction can make when it is combined with a useful product.

Instead of using 3D only as decoration, I wanted it to become part of the identity of the fitness platform.

The combination of React + React Three Fiber + Three.js + AI gave me a strong foundation for experimenting with interactive web experiences.

I'm continuing to improve ASFORGE FITNESS and explore how AI and 3D can work together to create better fitness experiences.

🔗 Technologies Used

Frontend: React, Vite, JavaScript

3D: Three.js, React Three Fiber, Drei

3D Assets: GLB models and animations

Effects: React Three Postprocessing, Bloom, particles and lighting

AI: Gemini

If you're also experimenting with React Three Fiber, I'd love to hear what kind of 3D experiences you're building.

#react #threejs #javascript #webdev

How I Built a Multilingual AI Fitness Coach with Gemini and 90+ Languages

Aman singh — Wed, 23 Sep 2026 15:33:00 +0000

How I Built a Multilingual AI Fitness Coach with Gemini and 90+ Languages

Building an AI fitness coach is one thing.

Building one that can communicate with users across 90+ languages, understand fitness-related questions, respond naturally, and also support voice interaction was a much more interesting challenge.

For my fitness project ASFORGE FITNESS, I wanted to go beyond a traditional chatbot.

So I built Titan AI Coach — a multilingual AI fitness assistant powered by Google Gemini, React, JavaScript, and a Node.js backend.

In this article, I'll explain how I approached the multilingual AI system and how the different parts work together.

🤖 What is Titan AI Coach?

Titan AI Coach is the AI assistant inside my fitness platform.

The goal was to make it feel more like a digital fitness coach rather than a simple text chatbot.

It can help users with things like:

🏋️ Workout guidance
🥗 Nutrition-related questions
💪 Exercise guidance
📋 Fitness routines
💬 General fitness conversations
🌍 Multilingual conversations
🎙️ Voice interaction

The project combines an interactive frontend with a backend AI service.

🌍 Why 90+ Languages?

One of the biggest ideas behind this project was accessibility.

A fitness assistant shouldn't be limited to English users.

Different users are comfortable communicating in different languages, so I wanted Titan to support a large number of languages.

The application maintains a language list and allows the user to select their preferred language.

The selected language becomes part of the AI interaction so that the response can be generated in the user's chosen language.

For example, a user can interact using languages such as:

English
Hindi
Spanish
French
German
Japanese
Portuguese
Italian

…and many more.

The exact supported language list is maintained by the application rather than hard-coding a small number of languages.

🧠 Using Gemini as the AI Brain

For the AI responses, I used Google Gemini.

The basic architecture looks like this:

User
↓
React Frontend
↓
Titan AI Coach
↓
Node.js / Express Backend
↓
Google Gemini API
↓
AI Response
↓
React UI

The important part is that the Gemini API key is not exposed directly in the frontend.

Instead, the frontend communicates with my backend, and the backend communicates with Gemini.

This keeps the API credential on the server side.

⚛️ React Frontend

The frontend is built with React.

The AI Coach UI handles:

User messages
AI responses
Language selection
Chat history
Loading states
Voice interaction
Error handling

The frontend sends the user's request to the backend instead of directly calling Gemini.

A simplified version of the flow looks like:

const response = await fetch("/api/chat", {
method: "POST",
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify({
message,
language
})
});

const data = await response.json();

The backend then processes the request and sends the AI response back to React.

🟢 Node.js + Express Backend

I used Node.js with Express as the backend layer.

The backend is responsible for:

Receiving the user's message
Receiving the selected language
Preparing the Gemini request
Calling the Gemini API
Handling errors
Returning the response to the frontend

This separation also makes the application easier to maintain.

Instead of putting everything inside the React application, the AI logic stays behind the backend API.

🌐 Language Switching

The user can select a language from the AI Coach interface.

The selected language is then passed to the backend.

Conceptually:

Selected Language
↓
User Message
↓
Backend
↓
Gemini
↓
Response in Selected Language

This means the same AI Coach can be used by users with different language preferences without creating a separate chatbot for every language.

🎙️ Adding Voice Interaction

I also wanted Titan to support voice interaction.

For this, the browser's speech capabilities are used for voice input and speech output.

The basic experience is:

User speaks
↓
Speech Recognition
↓
Text
↓
Gemini AI
↓
AI Response
↓
Speech Synthesis
↓
User hears response

This makes the interaction more natural than typing every question.

It also makes the AI Coach feel closer to an actual digital assistant.

🔄 Retry and Fallback Handling

AI APIs can sometimes fail because of temporary network problems, rate limits, or service issues.

Instead of immediately showing an error to the user, I added retry and fallback handling to make the experience more reliable.

The general flow is:

Request
↓
Gemini API
↓
Success? ── Yes → Return Response
│
No
↓
Retry
↓
Success? ── Yes → Return Response
│
No
↓
Fallback / Error Message

This is especially useful for a user-facing application where a temporary API issue shouldn't completely break the interface.

🔐 Keeping API Keys Secure

One of the most important lessons while building this project was never exposing private API keys in frontend code.

For example, sensitive keys should not be hard-coded like this:

const API_KEY = "YOUR_SECRET_KEY";

Instead, sensitive credentials are stored in environment variables on the backend.

For example:

GEMINI_API_KEY=your_secret_key

The frontend talks to the backend, and the backend uses the secret key.

This is much safer than exposing the key to every browser user.

🏋️ Making the AI Fitness-Specific

Titan isn't designed as a completely generic chatbot.

The project is a fitness platform, so the AI experience is designed around fitness-related conversations.

The surrounding website provides context through features such as:

Workout routines
Diet routines
Exercise demonstrations
Fitness tracking
Premium fitness plans
AI coaching

The idea is to make the AI part of the complete fitness experience instead of having an unrelated chatbot sitting on a fitness website.

🧩 Technologies Used

Here are the main technologies behind this project:

Technology Purpose
React Frontend UI
JavaScript Application logic
Vite Frontend development/build
Node.js Backend runtime
Express API server
Google Gemini AI responses
Browser Speech APIs Voice interaction
Three.js 3D/interactive experience
Supabase Application data/backend services
🚧 Challenges I Faced

Building this system wasn't just about connecting an AI API.

Some of the challenges included:

Multilingual responses

Supporting many languages required a clean way to manage language selection and pass that information through the AI request.

API failures

Temporary failures required retry and fallback handling.

Voice interaction

Speech recognition and speech synthesis behave differently across browsers, so voice features need careful handling.

Security

Keeping API credentials away from frontend code was another important part of the architecture.

User experience

The AI shouldn't feel like a plain API response.

The goal was to make Titan feel integrated with the rest of the fitness platform.

🔮 What's Next?

The project is still evolving.

Some of the areas I'm working toward include:

Better AI fitness conversations
More personalized coaching
Improved voice interaction
Better workout recommendations
User progress integration
More intelligent fitness guidance
Further improvements to the multilingual experience
Mobile application expansion

The long-term goal is to turn the project into a more complete AI-powered fitness platform.

💡 What I Learned

This project taught me that building an AI application is not only about calling an AI API.

A good AI product also needs:

A clean frontend
Secure backend architecture
Error handling
Good user experience
Voice interaction
Language support
Reliable API communication
Proper separation between frontend and backend

The AI is only one part of the complete system.

🚀 Final Thoughts

Titan AI Coach started as an idea to add an AI assistant to my fitness website.

It gradually became a much bigger experiment involving AI, multilingual communication, voice interaction, React, Node.js, Three.js and backend architecture.

The most interesting part for me was seeing a fitness assistant communicate across many different languages while remaining part of the same application.

I'm continuing to improve ASFORGE FITNESS and experiment with new ways to combine AI with fitness technology.

Thanks for reading! ❤️

If you're also building AI-powered applications, I'd love to hear what you're working on.

🔗 Project

ASFORGE FITNESS — AI-Powered Fitness Platform

Built with React • JavaScript • Gemini AI • Node.js • Three.js

How I Added Razorpay Payments to My React Fitness Website

Aman singh — Tue, 15 Sep 2026 15:32:14 +0000

Building a fitness website is not only about workouts, animations, and a good-looking UI.

If you want to turn a fitness platform into a real product, you also need a way for users to purchase memberships.

While building my React-based fitness website, I decided to integrate Razorpay so users could join different membership plans directly from the website.

In this article, I'll show how I implemented the payment flow, connected React with a Node.js backend, created Razorpay orders securely, and verified payments on the server.

The project uses:

React + Vite
Node.js + Express
Razorpay
Supabase
Vercel

The goal was to create a simple flow:

Choose Plan → Create Order → Open Razorpay Checkout → Complete Payment → Verify Payment

1. Setting Up the Membership Plans

For the fitness website, I created three simple membership plans:

Plan	Price
Basic	₹99
Premium	₹149
Elite	₹199

Each plan has its own Join Now button.

When a user clicks the button, the frontend sends the selected plan to the backend instead of directly trusting the price from the browser.

This is important because payment amounts should be controlled on the server side.

For example, my backend keeps the plan prices like this:

const MEMBERSHIP_PLANS = {
  basic: {
    name: "Basic",
    amount: 9900
  },
  premium: {
    name: "Premium",
    amount: 14900
  },
  elite: {
    name: "Elite",
    amount: 19900
  }
};

## 2. Creating a Razorpay Order on the Backend

I did not create the Razorpay order directly from the React frontend.

Instead, the frontend sends the selected plan to my Node.js + Express backend.

The backend then creates the Razorpay order using the server-side amount configured for that plan.

The basic flow looks like this:

text
React Frontend
↓
Select Membership
↓
POST /api/create-order
↓
Node.js + Express
↓
Razorpay API
↓
Order ID
↓
Razorpay Checkout

app.post("/api/create-order", async (req, res) => {
try {
const { planId } = req.body;

const plan = MEMBERSHIP_PLANS[planId];

if (!plan) {
  return res.status(400).json({
    error: "Invalid membership plan"
  });
}

const razorpay = getRazorpayClient();

const order = await razorpay.orders.create({
  amount: plan.amount,
  currency: "INR",
  receipt: `receipt_${Date.now()}`
});

res.json({
  orderId: order.id,
  amount: order.amount,
  currency: order.currency,
  planName: plan.name
});

} catch (error) {
console.error(error);
res.status(500).json({
error: "Unable to create order"
});
}
});

3. Opening Razorpay Checkout from React

Once the backend creates the order, it returns the Razorpay orderId to the React frontend.

The frontend then uses the Razorpay Checkout SDK to open the payment window.

The flow is:

User clicks "Join Now"
        ↓
React calls /api/create-order
        ↓
Backend creates Razorpay Order
        ↓
Frontend receives Order ID
        ↓
Razorpay Checkout opens
        ↓
User selects payment method
        ↓
Payment is completed

const handlePayment = async (planId) => {
  const response = await fetch("/api/create-order", {
    method: "POST",
    headers: {
      "Content-Type": "application/json"
    },
    body: JSON.stringify({ planId })
  });

  const order = await response.json();

  const options = {
    key: import.meta.env.VITE_RAZORPAY_KEY_ID,
    amount: order.amount,
    currency: "INR",
    name: "Official ASForge",
    description: `${order.planName} Membership`,
    order_id: order.orderId,

    handler: async (paymentResponse) => {
      // Payment verification happens here
    }
  };

  const razorpay = new window.Razorpay(options);
  razorpay.open();
};

## 4. Verifying the Payment Securely

Opening the Razorpay Checkout is only one part of the payment process.

After a successful payment, Razorpay returns payment details such as:

- `razorpay_payment_id`
- `razorpay_order_id`
- `razorpay_signature`

I send these values back to my backend for verification.

The backend generates an HMAC SHA256 signature using the Razorpay Key Secret and compares it with the signature received from Razorpay.

A simplified implementation looks like this:

js
app.post("/api/verify-payment", (req, res) => {
try {
const {
razorpay_order_id,
razorpay_payment_id,
razorpay_signature
} = req.body;

const generatedSignature = crypto
  .createHmac("sha256", process.env.RAZORPAY_KEY_SECRET)
  .update(
    `${razorpay_order_id}|${razorpay_payment_id}`
  )
  .digest("hex");

const isValid =
  generatedSignature === razorpay_signature;

if (!isValid) {
  return res.status(400).json({
    success: false,
    error: "Invalid payment signature"
  });
}

res.json({
  success: true,
  message: "Payment verified successfully"
});

} catch (error) {
console.error(error);

res.status(500).json({
  success: false,
  error: "Payment verification failed"
});

}
});

Step 8 — Keeping Secrets Safe

## 5. Keeping Razorpay Secrets Safe

One of the most important parts of the integration was keeping the Razorpay credentials secure.

I used environment variables for the backend credentials:

env
RAZORPAY_KEY_ID=your_key_id
RAZORPAY_KEY_SECRET=your_secret
VITE_RAZORPAY_KEY_ID=your_key_id

Step 9 — Complete Payment Flow

## 6. The Complete Payment Flow

After putting everything together, the complete membership flow looks like this:

text
User
↓
Selects a Plan
↓
Clicks "Join Now"
↓
React Frontend
↓
POST /api/create-order
↓
Express Backend
↓
Razorpay API
↓
Order ID
↓
Razorpay Checkout
↓
User Pays
↓
Payment Details
↓
POST /api/verify-payment
↓
Server-Side Verification
↓
Payment Success

Step 10 — Supabase

## 7. Connecting Payments with Supabase

After getting the Razorpay payment flow working, I also connected Supabase to the project.

Supabase can be used to store application data such as:

- User information
- Memberships
- Payment records
- Workout plans
- Diet plans
- Progress
- AI usage
- Website events

For example, after a verified payment, a payment record can be stored with information such as:

text
user_id
plan
amount
razorpay_order_id
razorpay_payment_id
payment_status
created_at

Step 11 — Vercel Deployment

## 8. Deploying the Payment System

My frontend is built with Vite and deployed using Vercel.

During local development, the React application communicates with my local Express server.

For production, the payment endpoints need to be available from the deployed application as well.

The production architecture can look like:

text
React + Vite
↓
Vercel
↓
/api/create-order
/api/verify-payment
↓
Razorpay

Step 12 — Problems I Faced

## 9. Problems I Faced During Integration

While integrating Razorpay, I ran into a few common issues.

### Duplicate function declarations

While modifying the React component, I accidentally declared the same `handlePayment` function more than once.

This resulted in an error similar to:

text
Identifier 'handlePayment' has already been declared

11. What I Learned

This project taught me that adding payments to a website is more than simply opening a checkout window.

The important concepts I learned were:

Never trust payment amounts coming directly from the frontend.
Create orders on the backend.
Keep secret keys on the server.
Verify Razorpay signatures server-side.
Use environment variables for sensitive credentials.
Separate development and production API configurations.
Store payment records only after successful verification.
Think about security while designing the payment flow, not after finishing the UI.

What's Next?

In the next part, I'll explore how I built a multilingual AI fitness coach that can interact with users in multiple languages using Gemini.

Thanks for reading! 💪

If you enjoyed the article, feel free to follow me for more posts about:

React
AI
Three.js
Full-stack development
Fitness technology

Build. Train. Improve.

— Aman Singh

How I Built an AI Fitness Coach with Gemini, React and Three.js

Aman singh — Mon, 07 Sep 2026 10:43:39 +0000

How I Built an AI Fitness Coach with Gemini, React and Three.js

Building a website is one thing. Building a website that can actually interact with users is something completely different.

While working on my fitness project under ASForge, I wanted to create more than a traditional fitness website. I wanted to build an AI fitness coach that could communicate with users, answer questions, and feel like an actual character inside the website.

That's how Titan was created — an AI-powered fitness coach built with React, Three.js and Gemini.

The Idea Behind Titan

The goal was simple:

Create a fitness website where the AI coach is not just a chatbot sitting inside a box, but a 3D character that interacts with the user.

Titan was designed to provide an interactive experience where users can:

Ask fitness-related questions
Chat with the AI
Use voice interaction
Receive spoken responses
Switch between different languages
Interact with a 3D AI character

The project also gave me an opportunity to combine frontend development, 3D graphics and generative AI in one application.

Technologies I Used

The main technologies behind the project are:

React — User interface and application logic
Vite — Development environment and build tool
Three.js — 3D graphics
React Three Fiber — Using Three.js inside React
@react-three/drei — Useful Three.js helpers
Gemini API — AI responses
Web Speech API — Voice recognition and text-to-speech
JavaScript — Application logic and interactions

The combination of these technologies made it possible to create both the visual and conversational parts of Titan.

Creating the 3D AI Coach

The first challenge was getting a 3D character into the React application.

I used a GLB model for Titan and loaded it using useGLTF.

A simplified version looks like this:

import { useGLTF } from "@react-three/drei";

function Robot() {
const { scene } = useGLTF("/models/ai-coach.glb");

return (
object={scene}
scale={1.35}
position={[0.15, -0.85, 0]}
/>
);
}

I then placed the model inside a React Three Fiber Canvas.

<Canvas
camera={{
position: [0, 0.1, 3.8],
fov: 40,
}}

position={[5, 5, 5]}
intensity={2}
/>

Getting the correct camera position, model scale and character placement took quite a bit of experimentation.

A model can look perfect in a 3D viewer but appear too small, too large or partially outside the screen when placed inside a real webpage.

Adding Animation

The character should not feel like a static image.

The GLB model contains animations, so I used useAnimations to play them.

const { scene, animations } =
useGLTF("/models/ai-coach.glb");

const { actions } =
useAnimations(animations, group);

useEffect(() => {
if (!actions) return;

Object.values(actions).forEach((action) => {
action.reset().fadeIn(0.5).play();
});
}, [actions]);

This made Titan feel much more alive inside the website.

Connecting Titan to Gemini

The next part was giving Titan a brain.

Instead of hardcoding responses, I connected the application to the Gemini API.

The basic flow is:

User
↓
Titan UI
↓
React
↓
Backend API
↓
Gemini
↓
AI Response
↓
Titan

I kept the Gemini API key on the server side instead of exposing it directly in the React frontend.

The frontend sends the user's message to my backend:

const response = await fetch("/api/chat", {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
message: text,
language: language.name,
}),
});

The backend then communicates with Gemini and returns the generated response.

This separation is important because API keys should not be placed directly inside frontend code.

Giving Titan a Voice

A normal chatbot only shows text.

I wanted Titan to actually speak.

For that, I used browser speech capabilities.

The browser's speech synthesis API can convert text into spoken audio:

const utterance = new SpeechSynthesisUtterance(text);

utterance.rate = 0.88;
utterance.pitch = 0.55;
utterance.volume = 1;

speechSynthesis.speak(utterance);

Now the interaction becomes:

User asks a question
↓
Gemini generates the answer
↓
Titan receives the answer
↓
Speech Synthesis speaks it

The exact available voices depend on the user's browser and operating system, so voice characteristics can vary between devices.

Voice Input

I also wanted users to be able to talk to Titan instead of typing everything.

The browser's Speech Recognition API can convert speech into text.

The basic concept is:

const recognition =
new SpeechRecognition();

recognition.lang = "en-IN";

recognition.onresult = (event) => {
const transcript =
event.results[0][0].transcript;

console.log(transcript);
};

recognition.start();

The recognized text can then be sent to Gemini just like a normal typed message.

Handling Multiple Languages

Another goal of Titan was making the experience more accessible to users who are more comfortable communicating in Indian languages.

The interface includes language selection for languages such as:

Hindi
English
Bengali
Telugu
Tamil
Marathi
Gujarati
Kannada
Malayalam
Punjabi
Odia
Assamese
Urdu
Nepali
Konkani
Maithili
Sanskrit
Kashmiri
Sindhi
Dogri
Manipuri
Santali

The selected language is passed along with the user's request so Titan can respond in the selected language.

However, speech recognition and available voices are dependent on browser and device support, so multilingual voice behavior can vary.

Making Titan Feel Like an Assistant

One of the most interesting parts of the project was designing the interaction.

I didn't want the AI to behave like a simple form:

Question → Answer

Instead, I wanted the experience to feel more like talking to an assistant.

The interaction became:

Speak
↓
Speech Recognition
↓
Gemini
↓
Titan Response
↓
Text-to-Speech
↓
User

I also worked on interruption handling so that when the user genuinely starts another question while Titan is speaking, the current speech can be stopped and the new request can be processed.

This part was more difficult than I initially expected because the microphone can sometimes pick up the computer's own speaker output.

Using microphone echo cancellation and noise suppression helped, but the behavior can still depend on the hardware and environment.

Designing the AI Interface

I wanted the AI controls to stay separate from the 3D character.

The basic structure became:

┌─────────────────────────────┐
│ Titan AI Coach │
│ │
│ Chat / Talk / Language │
│ │
│ Messages │
│ │
│ 3D Titan │
│ 🤖 │
└─────────────────────────────┘

The 3D character is rendered in its own layer while the chat and controls are placed above or behind it using CSS positioning and z-index.

This creates the impression that Titan is actually standing inside the interface rather than being another ordinary HTML element.

Problems I Faced

The project wasn't straightforward.

Some of the biggest challenges were:

3D Model Positioning

The character was sometimes too small or moved outside the visible area.

I had to adjust:

Camera position
Field of view
Model scale
Model position
Canvas dimensions

Voice Interruption

Speech recognition could sometimes detect audio coming from Titan itself.

This created unexpected interruptions.

I experimented with:

Echo cancellation
Noise suppression
Automatic gain control
Microphone monitoring
Speech recognition confirmation

API Availability

AI APIs can occasionally return temporary errors or become unavailable during periods of high demand.

This made retry handling and proper error messages important.

Keeping API Keys Secure

I initially focused on making the AI work, but quickly realized that API credentials should never be exposed in frontend code.

Moving the API call to a backend was an important architectural improvement.

What I Learned

This project taught me that building an AI application isn't only about calling an AI API.

There are several layers involved:

UI/UX
+
3D Graphics
+
AI
+
Voice
+
Backend
+
Browser APIs

Each layer has its own challenges.

Three.js taught me about cameras, lighting, models and 3D positioning.

React taught me how to manage application state and UI.

Gemini introduced me to integrating generative AI into an actual application.

The Web Speech API showed me how powerful browser capabilities can be for voice interfaces.

And debugging all of these systems together taught me perhaps the most important lesson:

Building real products involves solving problems that don't appear in tutorials.

What's Next for Titan?

Titan is still a work in progress.

Some of the things I want to explore next include:

Better conversational memory
More personalized workout guidance
Improved voice interaction
Better multilingual support
Fitness progress tracking
More natural AI conversations
More advanced 3D character interactions
Better mobile experience

The long-term goal is to turn Titan from an AI feature into a complete AI fitness experience.

Final Thoughts

What started as a fitness website became an experiment in combining AI, 3D graphics, voice interaction and modern web development.

I'm building these projects under ASForge, where my goal is to explore and create practical AI-powered technology.

Titan is just one step in that journey.

If you're also building something with React, AI or Three.js, I'd love to hear about it.

Keep building. Keep learning. Keep experimenting. 🚀