The Project: Introducing AnemiaSense AI
In modern healthcare, accessibility is everything. Standard screenings for conditions like anemia—a shortage of healthy red blood cells—traditionally require invasive blood draws, specialized lab equipment, and days of waiting for results.
AnemiaSense AI was designed to change this. It is a full-stack medical diagnostic web application that analyzes a simple, non-invasive photograph of a patient’s palm to predict the likelihood of anemia.
At the core of the application is a custom Convolutional Neural Network (CNN) trained using TensorFlow and Keras. The model acts as a digital set of eyes, scanning the unique color distributions, pale skin tones, and creases of the palm to output a diagnostic prediction.
However, as the application transitioned from a local development script to a live, cloud-hosted service, it encountered a classic software engineering roadblock: computational latency.
The Problem: The Latency and Resource Cost of AI Inference
In software development, we refer to running an active machine learning model as inference. Unlike a simple web request that displays a profile page or fetches text from a database, inference is a massive mathematical calculation.
When a user uploads an image, the server must:
- Downsample the image into a uniform $256 \times 256$ pixel matrix.
- Normalize the color values.
- Pass those arrays through multiple layer weights, executing millions of matrix multiplications to calculate a prediction.
In production, this entire process took upwards of 300 milliseconds per request.
For a single user, 300ms feels like a minor lag. But at scale, this latency poses severe engineering challenges:
- Wasted Compute: Users frequently re-upload the same photo, refresh their screens, or double-tap buttons. Running heavy model math over the exact same pixels repeatedly wastes expensive server CPU and GPU cycles.
- Server Congestion: If multiple users hit the API at the same time, the server's processor spikes to 100%, causing incoming requests to queue up, time out, and crash.
- Sluggish User Experience: Modern web applications are expected to react instantly. A 300ms delay degrades the feel of the user interface.
To scale the app, we needed to make it smarter. We needed a way to remember past calculations.
The Architectural Solution: MD5 Hashing + Redis
To prevent our AI model from recalculating results it had already solved, we implemented a caching layer. Caching is the process of storing previously calculated results in a temporary, high-speed storage location so future requests can be served instantly.
Our caching architecture relies on two key technologies:
1. MD5 Cryptographic Hashing (The "Digital Fingerprint")
Comparing raw images pixel-by-pixel to see if they are identical is slow and memory-intensive. Instead, we use MD5 Hashing. An MD5 algorithm takes the binary data of an image file and compresses it into a unique, fixed-length 32-character string (e.g., 8fbe12c9...).
Think of this hash as a unique digital fingerprint. If even a single pixel in the image changes, the fingerprint changes completely. This fingerprint serves as our lookup key.
2. Redis (The "High-Speed Memory Scratchpad")
Redis (Remote Dictionary Server) is an in-memory database. While traditional databases write data to physical hard drives (which is relatively slow), Redis stores data directly in the server’s RAM. Because reading from RAM is incredibly fast, Redis can retrieve data in under 2 milliseconds.
By storing our digital fingerprints and their corresponding AI predictions in Redis, we built a lookup loop: when an image is uploaded, we check the cache. If we have the fingerprint on file (a Cache Hit), we bypass the neural network entirely.
System Architecture Flow
Here is how a palm image travels through the AnemiaSense API:
Implementation Deep Dive: The Code
The implementation is written in Python using Flask and the Redis client library. It is designed to be resilient, performant, and self-cleaning.
1. Graceful Connection Resilience
In system design, fallback options are vital. If the Redis server goes offline, the web application must not crash. The code below attempts a connection and, if it fails, falls back gracefully to running the application without cache support:
import os
import redis
# Initialize external Redis Cache connection
redis_url = os.environ.get("REDIS_URL", None)
redis_client = None
if redis_url:
try:
redis_client = redis.from_url(redis_url)
redis_client.ping() # Test if the database is responding
print(" Distributed Cache initialized: Connected to Redis successfully.")
except Exception as e:
print(f" Warning: Failed to connect to Redis. Running without cache. Error: {e}")
redis_client = None
else:
print(" No REDIS_URL environment variable found. Cache layer disabled by default.")
2. The Intercept, Hashing, and Lookup Loop
Inside our Flask /predict endpoint, we read the raw incoming bytes of the file, generate the MD5 hash key, query Redis, and return immediately if there is a match:
import hashlib
import json
from flask import jsonify
# 1. Read bytes and generate the cryptographic hash key
file_bytes = file.read()
file_hash = hashlib.md5(file_bytes).hexdigest()
# Crucial: Reset the file stream pointer so it can still be saved to disk on a miss
file.seek(0)
# 2. Check the Cache (Redis RAM)
if redis_client:
cached_result = redis_client.get(file_hash)
if cached_result:
print(f" CACHE HIT! Bypassing inference model for hash: {file_hash}")
data = json.loads(cached_result)
return jsonify({
'prediction': data['prediction'],
'probability': round(data['probability'] * 100, 2),
'cached': True # Informs the frontend UI to display the cache badge
})
Note on the Stream Pointer: Calling file.read() consumes the image data. We must run file.seek(0) to reset the file's internal pointer back to the beginning. If we forget this, any attempt to save the image for the model on a cache miss will result in a corrupted, empty file.
3. The Caching Write Loop (Cache Miss)
If the fingerprint is not found in Redis, the system saves the image, runs the Keras model, deletes the temporary file to preserve disk space, and saves the new result to Redis with a 24-hour expiration time (Time-To-Live, or TTL):
# 3. Cache Miss: Execute Keras Model Inference
file.save(filepath)
predicted_class, probability = predict_image(filepath)
# Remove the temporary image from disk
if os.path.exists(filepath):
os.remove(filepath)
# 4. Save the result to Redis (Expired automatically after 24 hours)
if redis_client:
r_payload = json.dumps({'prediction': predicted_class, 'probability': probability})
redis_client.setex(file_hash, 86400, r_payload) # 86,400 seconds = 24 hours
Quantifying the Impact
By shifting the computational load from CPU/GPU-intensive model calculations to lightweight memory lookups, we achieved substantial optimization:
| Performance Metric | Without Redis (Cache Miss / First Run) | With Redis (Cache Hit) | Improvement |
|---|---|---|---|
| Response Latency |
~300 ms to ~320 ms
|
~2 ms |
99.3% Latency Reduction |
| Time Complexity | $O(N)$ (CNN layer-by-layer forward pass) | $O(1)$ (Constant-time hash lookup) | Infinite scalability benefit |
| Resource Overhead | High CPU utilization spike | Negligible (microsecond RAM fetch) | Lower server hosting costs |
| User Experience | Slight load delay | Instantaneous display | Lightning-fast interface |
On the frontend, when the JavaScript UI receives 'cached': true from the backend, it dynamically displays a (Cache Hit) badge next to the diagnosis. This visually demonstrates the performance optimization directly to the user.
Enterprise Parallels: How Tech Giants Scale
While caching a custom CNN is a valuable project-level optimization, this exact system design pattern is what keeps the internet's most popular platforms online:
- Google Search Autocomplete: When you type into Google's search bar, it suggests terms instantly. Google cannot search its entire web index database for every keystroke. Instead, they cache popular autocomplete results in high-speed, in-memory databases like Redis. Common query results are served directly from RAM in milliseconds.
- Twitter / X Timelines: Twitter doesn't run a complex query to search millions of accounts and find recent posts every time you refresh your feed. They compile and cache active user timelines directly in Redis, delivering your home feed instantly.
- Netflix Session Tracking: When you pause a show on your TV and open it on your phone, Netflix knows the exact second you stopped. They use Redis to track and sync live player coordinates and session data in real-time.
Advanced Considerations: Edge Cases & Trade-offs
A professional system design requires understanding the trade-offs of your choices:
- Storage Management (RAM limits): Redis RAM is expensive. Storing raw images in Redis would fill up the memory and crash the server. We solved this by generating an MD5 hash, caching only the lightweight prediction string (a few bytes) rather than the original image data.
- Cryptographic Collision Risk: While MD5 is fast, it has a tiny risk of "collisions" (two different images generating the same hash). In a mission-critical clinical setting, swapping MD5 for SHA-256 offers a much larger keyspace to eliminate collision risk with minimal CPU overhead.
- Data Drift & Eviction: AI models are updated over time. If we train a new version of our CNN, the old cached predictions in Redis will become inaccurate. Setting a 24-hour TTL ensures that stale results naturally expire, while updating the cache key prefix on model redeployment guarantees that old data is never served.
Conclusion
System design is the art of choosing the right tool for the job. By placing an in-memory Redis cache in front of a deep learning model, we decoupled our application's response time from the complexity of the neural network. Caching is a simple, high-impact pattern that saves hosting costs, scales backend infrastructure, and provides a snappy, seamless user experience.
How do you manage latency in your machine learning deployments? Let’s connect and discuss in the comments!
Top comments (0)