Welcome to the final phase of development for MedReachAI. As we transition into the Software Integration phase, my partner, Collin, and I are shifting our focus from building core features to hardening our application for real-world usability, security, and stability.
This week, we tackled a massive milestone: getting our React frontend and FastAPI backend to talk to each other flawlessly under heavy data loads. It hasn't been without its roadblocks, but resolving those bottlenecks is exactly what this phase is about.
The Problem: Memory Crashes and the Integration Gap
MedReachAI is designed to ingest, clean, and analyze massive healthcare provider datasets. Previously, when we attempted to push large CSV files directly through our web server, we hit immediate bottlenecks. Cloud Run payload limits and Out of Memory (OOM) container crashes stopped our pipeline in its tracks.
To solve this, we pivoted to a Direct-to-Bucket architecture. I provisioned a Google Cloud Storage (GCS) bucket and configured our FastAPI backend to generate time-boxed signed URLs. This allowed the client to upload files directly to the cloud, completely bypassing the web server. Once uploaded, a new asynchronous pipeline pulls the data and processes it in memory-safe 500-record chunks.
However, when we moved to End-to-End (E2E) testing to verify this new architecture, we hit a wall.
Root Cause Analysis: 404 Dashboard Errors: As the terminal output shows, the frontend UI was attempting to fetch data from /api/companies/acme/provider_walkthrough and /api/companies/acme/records. These endpoints were returning 404 Not Found because they hadn't been fully scaffolded on the FastAPI side yet.
The 403 Upload Block: Concurrently, the backend strictly requires an X-User-Role HTTP header to authorize the signed URL generation. The React frontend was reading the user's admin role from the UI state but failing to attach it to the network request. Without that header, the backend defaulted to a read-only "Viewer" state and blocked the upload.
The Solution: Sprint 7 and Handshake Stabilization
To properly manage this technical debt, we scoped out an Agile sprint specifically dedicated to resolving these cross-system integration bugs. For Sprint 7, we created a 15-story-point Epic: End-to-End (E2E) Integration & Handshake Stabilization.
Here is how we divided the workload to patch the pipeline:
Auth & Header Parity: Collin is updating the React fetch requests to explicitly pass the UI state role into the X-User-Role header, clearing the backend’s security policies and resolving the 403 upload rejection.
Endpoint Alignment: I am scaffolding and wiring up the missing /records and /provider_walkthrough API routes so the client can successfully fetch and render the pipeline's output metrics without hitting the 404s we saw in the server logs.
E2E Dynamic Load Testing: Once the handshake is secure, I will conduct a dynamic analysis load test, pushing a massive dataset through the application. We will monitor the backend memory consumption logs to verify that the 500-record chunking pipeline permanently resolves the previous OOM crashes.
Graceful UI Error States: Instead of a generic failure message, the UI will now gracefully handle network drops and explicitly warn users if they lack the required editor permissions.
Software integration is rarely just about making the code work; it’s about making the systems communicate securely and gracefully. With this sprint board locked in, we have a clear path to a stable, production-ready upload pipeline. Check back next week for the results of our load test.
Top comments (0)