In traditional sleep and meditation apps, the architecture is straightforward: pre-recorded MP3/AAC audio files are hosted on an S3 bucket and served globally via a CDN. While this approach is cost-effective and rock-solid, it suffers from a fundamental limitation: users quickly experience listening fatigue from repetitive scripts, and static audio cannot react to a user's real-time physical or emotional state.
To overcome this, our team developed WhatsFeel—an app designed for meditation, sleep, and daily relaxation that is powered entirely by artificial intelligence (LLM text generation + real-time TTS voice synthesis). Instead of looping fixed recordings, WhatsFeel features intelligent assistants (including Meditation, Sleep, Nature Sounds, and a completely free Q&A Emotional Chat agent) that dynamically generate guidance in real time based on user interactions and biometric signals.
However, moving to 100% real-time AI generation introduces significant server bandwidth overhead and latency challenges. In this article, I’ll share the engineering architecture behind WhatsFeel, focusing on our high-concurrency caching model, standard Bluetooth biofeedback integration, and low-latency streaming pipeline.
System Architecture Overview
The core engineering goal of WhatsFeel is to bridge software and physiological signals: transforming real-time heart rate data into dynamic LLM prompt context to generate "breathing, adaptive" audio guidance.
[ BLE Heart Rate Devices (Garmin / Coros / Polar) ]
│ (Standard GATT: 0x180D Service)
▼
[ Client Application (BLE Manager & Audio Queue) ]
│ (Calculates HR trends & smooths noise)
▼
[ Server-Sent Events (SSE) / WebSocket ]
│ (Sends HR state + Session Context)
▼
[ AI Orchestration Engine ]
│ (Dynamic Prompting & Caching Strategy)
▼
[ LLM Stream Generation ──> Real-Time TTS Pipeline ]
│
▼
[ Seamless Audio Stream Playback on Client ]
The system is split into four distinct layers:
Biometric Perception Layer: Collects live heart rate metrics from wearable straps/watches over standard BLE.
Client Layer: Handles Bluetooth packet parsing, heart rate trend smoothing, and chunked audio playback (Audio Queue).
Routing & Cache Scheduling Layer: Routes requests based on user subscription tiers and geographic/language attributes.
AI Core Layer: Handles dynamic prompt construction, LLM token streaming, and customized TTS voice synthesis.
Key Technical Challenges & Solutions
- Dynamic "Region + Language" Group Caching (Overcoming Bandwidth & LLM Bottlenecks) Since every session in WhatsFeel is generated by AI on the fly, traditional static CDN caching cannot be used. This creates massive challenges for server bandwidth and LLM API costs. To solve this without compromising on the feeling of real-time evolution, we built a tiered scheduling and dynamic sliding-window group cache:
Biometric VIP Pipeline: For users connected via Bluetooth heart rate devices, the system provisions an isolated pipeline. Content is generated 100% in real time, adapting directly to their heart rate fluctuations.
Group Sliding-Window Cache:
Group Routing: Non-biometric users are grouped by region, time zone, and language preferences.
Dynamic Reuse: Users within the same group share a rolling cache of recent AI-generated audio segments. This is not a static pre-recorded library, but a live "rolling pool" that constantly evolves on the server.
Architectural Balance: Users always hear fresh, evolving guidance whenever they open WhatsFeel, while server bandwidth and LLM computing costs are reduced significantly.
Free Tier (AI Chat Assistant): Includes a completely free Q&A emotional support agent, providing a low-barrier space for users to talk through late-night anxiety without requiring a subscription.
- Standard BLE 0x180D Protocol & Heart Rate Biofeedback To ensure broad hardware compatibility, we avoided proprietary Bluetooth protocols in favor of global standards:
Universal Hardware Support: By implementing the standard Bluetooth Heart Rate Service (0x180D), WhatsFeel works out-of-the-box with devices from Garmin, Coros, Polar, and any standard BLE heart rate chest strap or armband.
Biometric-Driven Prompt Conditioning: The client extracts heart rate values and converts them into LLM context parameters. If WhatsFeel detects a sudden heart rate spike due to stress or insomnia, the AI adjusts its prompt strategy on the fly—guiding the voice model to adopt a slower cadence and soothing tone to help regulate the user's pulse.
- Low-Latency Streaming Pipeline & Privacy Design Token-Level Chunking & Audio Queuing: As the LLM streams text tokens, the server uses a regex-based punctuation parser to split sentences at natural pause points (commas, periods). As soon as a clause is ready, it is immediately dispatched to the TTS engine. The client uses an audio queue to buffer and play audio chunks seamlessly, eliminating lag.
Privacy-First Architecture: WhatsFeel never stores user health or biometric data on cloud databases. Heart rate values exist strictly in ephemeral memory (RingBuffer) during the active session for real-time adjustments, ensuring strict data privacy compliance.
Conclusion
Building WhatsFeel taught us that combining generative AI (LLM + TTS) with IoT biometrics opens up a whole new paradigm for wellness applications. By leveraging standard 0x180D BLE protocols alongside a group sliding-window caching model, it's possible to deliver adaptive, non-repetitive AI experiences at scale.
I'd love to hear your thoughts! Feel free to ask questions in the comments about BLE packet parsing, TTS streaming optimization, or dynamic LLM prompt orchestration.
Top comments (0)