Monitoring isn't just about logging; it's about building observability that scales to handle (200KB to 3MB)/minute per user across multiple remote services, and provides the debugging capabilities needed for a financial trading platform where every millisecond matters.
We are not going into backend design here. Let me know if you would be interested in that as well.
Why This Matters
In a trading application, monitoring serves multiple critical purposes:
- Debugging: Real-time issue identification during development and production
- Audit Trail: Regulatory compliance for financial transactions
- Performance Monitoring: Latency tracking for trading operations
- Error Tracking: Proactive issue detection before users report problems
- Analytics: User behavior and system performance insights
The monitoring system must handle:
- Volume: (200KB to 3MB)/minute per user
- Reliability: Zero message loss, even during network failures
- Multi-Service: Simultaneous logging to Service-1 and Service-2
- Offline Support: Seamless operation without internet connectivity
- Caching: Shouldn’t burden app operations by continuously pushing logs
- Scale: 10,000+ concurrent users
Architecture Overview
┌─────────────────────────────────────────────────────────────┐
│ Application Layer │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Local │ │ Remote │ │ Logs │ │
│ │ Logging │ │ Logging │ │ UI Screen │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ Facade Layer │
│ ┌─────────────┐ ┌─────────────┐ │
│ │ Local │ │ Remote │ │
│ │ Logger │ │ Logger │ │
│ │ │ │ Facade │ │
│ └─────────────┘ └─────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ Coordination Layer (Ports) │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Remote Log Coordinator │ │
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │
│ │ │ Timer │ │ Cache + │ │ Sender │ │ Config │ │ │
│ │ │ │ │ Batching│ │ │ │ │ │ │
│ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ │
│ └─────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ Implementation Layer (Adapters) │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │ Native │ │ Hive │ │ HTTP │ │ Timer │ │
│ │ Logger │ │ Cache │ │ Client │ │ Service │ │
│ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ External Services │
│ ┌─────────────┐ ┌─────────────┐ │
│ │ Service 1 │ │ Service 2 │ │
│ │ (HTTP) │ │ (HTTP) │ etc │
│ └─────────────┘ └─────────────┘ │
└─────────────────────────────────────────────────────────────┘
`
Local Logging Architecture
Why Talker?
When evaluating logging solutions for the new app, we needed more than just console prints. We needed a production-ready, testable, and maintainable logging layer that fits clean architecture principles.
1. Production-Ready Debug UI
Talker gives us a built-in, zero-effort debug screen:
`
// Built-in debug screen with real-time filtering
Widget debugScreen() {
if (kReleaseMode) return const SizedBox.shrink();
return TalkerScreen(talker: _instance._talker);
}
Out of the box, we get:
- Real-time log streaming
- Log-level filtering (Debug, Info, Warning, Error)
- Search and export
- Performance metrics and insights
2. Zero Configuration
LoggerImpl._()
: _talker = TalkerFlutter.init(
settings: TalkerSettings(enabled: !kReleaseMode),
logger: TalkerLogger(
settings: TalkerLoggerSettings(enable: !kReleaseMode),
),
);
Talker handles release detection, formatting, performance optimization, and memory management internally.
3. Clean Architecture Integration
We abstracted Talker behind our own Logger interface:
abstract class Logger {
void init({required bool enabled});
void debug(String message);
void info(String message);
void warn(String message);
void error(String message, [Object? err, StackTrace? st]);
Widget debugScreen();
}
Benefits:
- Mock and unit-test logging in isolation
- Inject loggers seamlessly across layers
- Swap to another logger without touching business logic
Bottom line: Talker hit the sweet spot—powerful tooling with a minimal footprint, perfectly aligned with architectural principles.
Remote Logging Architecture
As part of a Flutter app, we designed a remote logging system that goes far beyond simple print() statements. It’s engineered to handle scale, offline scenarios, and multiple logging backends without sacrificing maintainability.
Design Principles
- Zero Message Loss — every log is guaranteed to be delivered
- Offline-First — logs queue and persist seamlessly without connectivity
- Multi-Service Support — simultaneous dispatch to multiple providers
- Scalable Batching — network-efficient, configurable batch sizes
- Clean Architecture — loosely coupled, testable, and maintainable components
Components
Remote Log Coordinator
The brain of the system:
- Routes logs (immediate vs cached)
- Manages connectivity state
- Coordinates periodic flushes
- Handles retries, errors, and recovery
- Owns cache lifecycle
class RemoteLogCoordinator {
final LogCache _cache;
final LogSender _sender;
final ConnectivityChecker _connectivity;
final PeriodicTimer _timer;
final ConfigManager _configManager;
final MessageBatcher _batcher;
}
Hive-Based Cache
- Mutex-protected for thread safety
- Survives restarts (zero data loss)
-
getAllAndClear()ensures atomic operations - Lazy loading keeps memory footprint low
Multi-Service Sender
- Parallel HTTP requests with timeouts
- Lazy service initialization
- Batch-level success/failure metrics
Message Batching
class MessageBatcherImpl implements MessageBatcher {
@override
int get maxBatchSizeBytes => 600 * 1024; // 600 KB
}
- 600 KB max per batch
- Oversized single messages isolated automatically
- Optimal grouping for throughput and latency
Remote Logging Flow
Scalability Considerations
1. Performance at Scale
- Throughput: ~3.3 MB/s sustained
- Batch Size: 600 KB (max)
- Send Interval: 3 minutes
2. Memory Management
- Lazy loading with Hive
- Chunked batch processing
- Automatic cleanup
- Configurable cache size
3. Network Efficiency
- Fewer HTTP calls → less overhead
- Larger payloads → better compression
- Connection reuse with keep-alive
- Per-batch timeouts to isolate failures
Error Handling & Resilience
- Graceful Degradation — fallback to cache if direct send fails
- Service Failure Isolation — one service failing doesn’t block others
- Cache Recovery — failed sends restored to cache for retry
Testing Strategy
- Unit Tests — mocked interfaces for coordination logic
- Integration Tests — real service contract validation
- Performance Tests — load simulation under production-like conditions
Security Considerations
- Authentication: HMAC-SHA256 signatures per request
- Data Privacy: no PII, enforced HTTPS
- Access Control: service-specific tokens
- Rate Limiting: batch caps and configurable intervals
Conclusion
This architecture ensures logs are:
- Reliable — Zero message loss with robust offline support
- Flexible — Multi-service logging via swappable interfaces
- Maintainable — Layered clean architecture with strong test coverage
- Observable — Real-time debug screens and deep system insights
We've struck the right balance between simplicity and power—an architecture that scales with app growth while keeping us confident in production diagnostics and rapid iteration.
Top comments (0)