DEV Community: Scott Shoemaker

Scaling the Cloud: Overcoming Infrastructure Hurdles in MedReach-AI (Sprint 2 Retrospective)

Scott Shoemaker — Mon, 09 Nov 2026 16:13:56 +0000

As we wrap up Sprint 2 of the MedReach-AI capstone, our primary goal was to connect our React frontend to the FastAPI backend and breathe life into our Analytics Dashboard. We wanted to take raw, sanitized Healthcare Professional (HCP) data and turn it into actionable, real-time insights—visualizing geographic coverage, specialty distributions, and data quality trends.

What we actually got was a masterclass in cloud infrastructure scaling, memory limits, and network resilience.

Here is a look at the technical hurdles our engineering team conquered this week to stabilize our production pipeline.

Hurdle 1: Bypassing Firebase Proxy Limits
Our first roadblock hit during the data ingestion phase. We initially routed our frontend requests through Firebase Hosting, but quickly discovered that Firebase enforces a strict 60-second timeout on proxy requests. For massive CSV uploads requiring intensive backend validation, 60 seconds simply wasn't enough. Our solution was to refactor our API client to bypass the proxy entirely, routing large production requests directly to our Google Cloud Run service URL while preserving our Firebase Bearer authentication and dynamic company scoping.

Hurdle 2: The Out-of-Memory (OOM) Crash
With the network path cleared, we pushed a stress-test dataset of 30,000 records through the system. The result? A 500 Internal Server Error.

Because MedReach-AI utilizes a heavy, CPU-bound NLP model (spaCy) for PII scrubbing and data standardization before writing to Firestore, processing 30,000 rows at once caused our Cloud Run container to hit its default memory limit and forcefully shut down. We solved this by scaling our container hardware—upgrading to 8GiB of RAM and extending the request timeout ceiling to a full 60 minutes, giving the AI the runway it needed to process the batch.

_Caption: Firebase console displaying the successfully populated records subcollection after resolving the Cloud Run Out of Memory (OOM) crashes by chunking the database writes.
_

Hurdle 3: Firestore gRPC Stream Timeouts
The data finally made it into the database, but when the Analytics Dashboard tried to read it back, it triggered a 504 Stream removed error. The backend was attempting to download all 30,000 records in one continuous stream, causing Google's gRPC network to drop the connection.

Caption: Cloud Run logs showing the container successfully stabilizing and traffic shifting after deploying the paginated Firestore read operations.

To ensure complete fault tolerance, we overhauled our FastAPI read operations. We engineered paginated Firestore queries using limit and start_after cursors. By fetching the data in batches of 1,000 records at a time and aggregating them safely in server memory, we completely eliminated the dropped connections.

Caption: The finalized Analytics Dashboard successfully fetching and rendering the 30,000-record dataset in production without 504 timeouts.

Looking Ahead: Sprint 3
With the data pipeline officially locked down, the Analytics Dashboard is now rendering beautifully in production. As we transition into Sprint 3, our focus shifts from core engineering to project packaging. We will be producing our commercial spot, finalizing our executive presentation, and preparing to defend the MedReach-AI architecture in our final capstone review.

Securing the Analytics Pipeline and Pivoting for Demo Day

Scott Shoemaker — Mon, 02 Nov 2026 14:16:18 +0000

Sprint Retrospective: Implementing Zero-Trust Analytics

The conclusion of Sprint 11 was entirely focused on access control and finalizing our live cloud deployments. Our primary challenge was refactoring the dashboard metrics endpoints to ensure they were accessible to any logged-in user—regardless of their specific role hierarchy (Admin, Editor, Viewer)—while maintaining strict security against unauthenticated traffic.

Instead of relying on rigid Role-Based Access Control (RBAC) for these macro-level insights, I overhauled the FastAPI backend to utilize a zero-trust require_authenticated_user dependency. By streamlining our Firebase ID token validation, we successfully opened up the quality trends and specialty distribution metrics to all valid users, ensuring any request without a verified token still hits a hard 401 Unauthorized wall. With these changes pushed through our GitHub Actions pipeline and successfully deployed to Google Cloud Run, our V1.0 backend architecture is officially locked in.

Caption: Live MedReach-AI dashboard successfully fetching the executive metrics via Cloud Run, bypassing RBAC while enforcing strict Firebase token validation.

Sprint 12 Planning: UI Polish and Demo Preparation

With our final Capstone presentation scheduled for November 19th, Sprint 12 requires a strategic pivot. As the Technical Project Manager, I am actively preventing late-stage scope creep by transitioning our focus from complex backend data aggregation to presentation polish and frontend stability. I have mapped out this week's Jira backlog to maximize our team velocity using AI-assisted coding tools.

Caption: Sprint 12 Backlog detailing our UI feature freeze, static demo data integration, and Capstone commercial production tasks.

Our core deliverables for this development cycle (Nov 2 – Nov 8) include:

Global Navigation UI (MA-95): Collin is building out the final interactive frontend components, including the top-right user profile dropdown, secure Firebase logout routing, and UI modals for system notifications and global search.

Analytics Demo Data Binding (MA-96): To guarantee a flawless presentation without risking database downtime, I am utilizing our VS agent to wire the newly built Analytics page to a local JSON payload of static, high-fidelity demo data.

Capstone Commercial Production (MA-97): I will be drafting the script and utilizing AI narration tools to produce our final 60-second marketing commercial showcasing the application's core value proposition.

V1.0 Code Freeze & Documentation (MA-98): I am enforcing a hard feature freeze on the main branch. My final task is conducting regression testing on our live production URLs and finalizing the repository documentation with our zero-trust architecture diagrams.

We will convene for our standard sync later this week to review the dropdown Pull Requests and review the commercial storyboard before Sunday's milestone submission.

Overcoming API Bottlenecks and Prepping for V1.0 Deployment

Scott Shoemaker — Mon, 26 Oct 2026 14:24:49 +0000

Sprint Retrospective: Unblocking the Integration Pipeline
The conclusion of our previous development cycle required a heavy focus on technical project management and API integration. The primary goal was establishing a flawless end-to-end connection between our React frontend and FastAPI backend, but we encountered several critical bottlenecks that threatened our timeline.

The first major hurdle was a persistent 401 Unauthorized error. As we transitioned from simulated frontend logins to actual Firebase Authentication, the local Emulator Suite and our FastAPI CORS middleware fell out of alignment, causing secure token validation to fail. Immediately after resolving the auth routing, we were hit with a 404 NOT_FOUND error. Our Google AI generative model endpoint had been deprecated, requiring us to dive into the Python backend and upgrade the pipeline to gemini-3.8-flash.

The final and most complex challenge was version control drift. Our repository had become severely fragmented, with the backend branch sitting 37 commits behind production. Acting as the Scrum Master and lead integration specialist, I executed a complex merge, manually resolving content conflicts within main.py to preserve our backend execution logic while seamlessly integrating Collin’s new UI components.

Caption: _Full-stack pipeline verified. The frontend successfully routes through the backend to the Gemini API, gracefully catching a 402 billing response rather than crashing.

Sprint 11 Planning: V1.0 Release Candidate Stabilization
With the backend infrastructure locked in and the Git repository unified, Sprint 11 marks our official transition into final UI polishing and cloud deployment preparation. I have autonomously built out our Jira backlog for the week and organized the upcoming tasks under our "V1.0 Release Candidate Stabilization" Epic.

Caption: Sprint 11 Backlog detailing the final deployment and QA tasks assigned across the team.

Our core deliverables for this development cycle (Oct 26 – Nov 2) include:

Frontend UI Polish (MA-91): Collin is finalizing the last remaining CSS corrections and layout adjustments to ensure responsive consistency across all views.

Code Review & Integration (MA-92): I will be reviewing the upcoming UI Pull Request and executing the final merge into the main deployment branch to ensure the NLP query engine remains stable.

Infrastructure Provisioning (MA-93): I am staging our live cloud architecture by provisioning a free-tier Render web service for the FastAPI backend and initializing Firebase Hosting for the frontend.

Full-Stack QA Regression (MA-94): Collin and I will conduct a final local integration test to verify data retrieval and error handling before we push the application live to the public.

We will convene for our standard Thursday sync to review the UI pull request and block out any remaining hurdles before Sunday's milestone deadline.

Follow along as we finalize our Capstone deployment over the coming weeks. For the other technical PMs and full-stack devs out there, how do you typically handle late-stage version control drift when integrating your final frontend components

Closing the Testing Window and Polishing the Production Environment

Scott Shoemaker — Mon, 19 Oct 2026 20:00:11 +0000

As we enter the final stretch of our capstone project, our focus has officially shifted from building core functionality to refining the user experience. This week marks a major milestone for MedReachAI: the official close of our usability testing window.

Synthesizing User Feedback

With the testing kits returned and the think-aloud sessions complete, our primary objective this sprint is synthesizing that raw qualitative data. We are currently compiling the survey results to identify recurring friction points. This UX research report will serve as our blueprint for the final UI adjustments over the coming weeks, ensuring that the changes we make are directly driven by actual user interactions rather than assumptions.

Finalizing the NLP Backend

On the backend, we spent the early part of this sprint troubleshooting our Natural Language Processing query engine integration. After successfully diagnosing and bypassing a local Firebase Auth emulator bottleneck, the FastAPI backend is fully communicating with the React frontend. Our next step is conducting a final code review and safely merging the NLP feature branches into the main deployment pipeline.

Tackling Technical Debt and UI Polish

With the core API endpoints successfully firing, we are actively auditing the frontend to eliminate technical debt. During the MVP phase, we relied heavily on placeholder metrics and hardcoded arrays to build out the UI. This week, we are conducting a global dummy data purge—stripping out those placeholders, wiring up the remaining dead buttons, and hiding components that aren't fully supported by backend routes.

Additionally, we are overhauling the global authentication state. We are replacing the static profile indicators with dynamic, context-aware components that read directly from Firebase Auth, complete with a fully functional session logout infrastructure.

As we inch closer to graduation next month, these quality-of-life updates are crucial. We are no longer just building features; we are locking down a production-ready application.

Extending Usability Testing and Tackling the NLP Engine

Scott Shoemaker — Mon, 12 Oct 2026 17:15:25 +0000

This week marks Sprint 9 for the MedReachAI project, and our primary focus is balancing active user testing with major architectural updates. As we push toward finalizing our MVP, we are learning firsthand how critical domain-specific user feedback is for building a reliable compliance tool.

Here is a breakdown of our progress, early testing insights, and what Collin and I are building this week.

Expanding the Testing Window
Our initial unmoderated usability testing phase brought in some fantastic early insights, but we wanted to ensure a more statistically significant sample size before locking in our final analysis. We have officially extended the testing window into a "Phase 2" for this sprint, and our professor is assisting by distributing the testing kit to a broader network of staff and students.

Our testing kit utilizes a hybrid data approach: a direct 9,999-record slice of the public National Downloadable File to simulate real-world processing, alongside smaller, purely synthetic datasets explicitly engineered to trigger edge cases like PII leaks and duplicate records.

Early Insights & UI Refinements
Even with the testing window still open, early feedback from a tester with a healthcare outreach background has already shaped our active Jira backlog. They uncovered two critical areas for improvement:

Algorithm Recalibration: The system is aggressively flagging standard, public provider data (like clinic addresses and business emails) as critical PII violations, which artificially lowers the dataset's overall health score. Collin will be adjusting the backend scrubbing logic this week to properly whitelist these public fields.

Resolving "Ghost Buttons": The tester noted that clicking action buttons like "Anonymize" or "Override" on the Data Review table provided no visual feedback. We are implementing toast notifications and immediate row state updates to resolve this UI friction.

Heatmap Readability: We also noticed an overlapping formatting issue on the Data Quality Heatmap's x-axis labels, which will be corrected this sprint with updated CSS configurations.

Building the Natural Language Query Engine

While monitoring the ongoing usability tests, we are diving into one of our heaviest full-stack features: the Natural Language Query engine. Currently, the query page is operating on a hardcoded mock dataset. To make this functional, we have split the feature into two sequential tickets:

Backend Translation Logic: Building a new Python API endpoint that integrates an LLM to translate plain English queries (e.g., "How many cardiologists are in Florida?") into executable Pandas dataframe filters against our processed dataset.

Frontend Wiring: Once the API contract is stable, we will strip out the mock timeouts in the React frontend, wire up the search bar, and dynamically map the returned JSON payload into the results table.

By separating the LLM integration from the UI wiring, we keep our sprint velocity predictable and ensure the frontend development isn't bottlenecked by the complexities of prompt engineering.

We are looking forward to closing out the usability testing later this week, compiling the final report, and pushing these new features to the live Firebase environment.

Crossing the Deployment Bridge: Cloud Run, CORS, and Preparing for User Testing

Scott Shoemaker — Mon, 05 Oct 2026 16:17:58 +0000

Getting a full-stack application running flawlessly on your local machine is a great feeling. Getting that same application to run flawlessly on the live internet is an entirely different beast.

This past week on MedReach AI, my teammate Collin and I tackled the deployment transition, moving our testing from the local emulator to a live production environment. We successfully deployed our FastAPI backend to Google Cloud Run and our React frontend to Firebase Hosting, but the journey wasn't without its hurdles.

Here is a look at the technical challenges we solved this week, and how we are preparing for our upcoming user testing phase.

The CORS Bouncer and the Cloud Run Fix
Right after our initial Firebase deployment, our frontend dashboard threw a wall of "Failed to fetch" errors under our executive metrics. The backend was alive, the frontend was live, but they were refusing to talk to each other.

The culprit was a classic CORS (Cross-Origin Resource Sharing) block. Our Google Cloud Run backend was acting like a strict bouncer, rejecting requests from our brand new Firebase URL because it wasn't on the approved list.

The Solution: We had to carefully strip out our local development overrides (like bypassing offline Firestore configurations) and update our main.py file to explicitly whitelist our Firebase Hosting origin in the FastAPI CORSMiddleware. By pushing these changes through our GitHub pipeline, Cloud Run automatically pulled the latest revision. Within minutes, the CORS block was lifted, and our live executive metrics populated perfectly.

Redesigning the Data Health Score UX
Once the data pipeline was connected, we realized we had a critical UX flaw on the Data Review screen.

Our system grades datasets across different categories, but the UI displayed the points as fractions (e.g., 20/20 or 21.3/25). At a glance, a perfect 25/25 score looked exactly like "25 unresolved errors out of 25," creating instant panic for the user. Furthermore, our PII/PHI (Personally Identifiable Information) scanner was acting purely as an advisory warning rather than deducting points from the overall data health score—a major oversight for a healthcare data platform.

The Solution: We overhauled the scoring logic and the UI. We rebalanced the 100-point total across five distinct 20-point categories, finally giving PII/PHI its own dedicated scoring bucket. If sensitive data leaks are detected, the dataset is actively penalized. On the frontend, we appended "pts" to the fractions (e.g., 20/20 pts) and grouped the record-level privacy detections under a dedicated tab. This small change completely removed the ambiguity between "points earned" and "flags detected."

Sprint Week 2: The "Think-Aloud" User Testing
With the production environment stable and the UI refined, this week is all about stepping back and letting fresh eyes tear it apart.

We are officially entering our user testing sprint. I have scheduled three Think-Aloud testing sessions with Melissa, Alex, and Aaron for October 7th through the 9th. Because they have not spent the last several weeks staring at this specific codebase, they are the perfect candidates to expose any assumptions we have made about the platform's intuitive flow.

During these sessions, I will be observing how they navigate the CSV upload process and whether the newly updated Data Review screen actually makes sense to a first-time user. Meanwhile, Collin is diving deep into end-to-end integration testing to ensure our backend handles payload mismatches without throwing 500-level errors.

Building software in a vacuum is dangerous. It’s time to see how MedReach AI holds up in the wild. I'll share the results of our testing sessions next week!

Week 1: Stabilizing the Handshake — Overcoming E2E Integration Roadblocks

Scott Shoemaker — Tue, 29 Sep 2026 16:43:47 +0000

Welcome to the final phase of development for MedReachAI. As we transition into the Software Integration phase, my partner, Collin, and I are shifting our focus from building core features to hardening our application for real-world usability, security, and stability.

This week, we tackled a massive milestone: getting our React frontend and FastAPI backend to talk to each other flawlessly under heavy data loads. It hasn't been without its roadblocks, but resolving those bottlenecks is exactly what this phase is about.

The Problem: Memory Crashes and the Integration Gap
MedReachAI is designed to ingest, clean, and analyze massive healthcare provider datasets. Previously, when we attempted to push large CSV files directly through our web server, we hit immediate bottlenecks. Cloud Run payload limits and Out of Memory (OOM) container crashes stopped our pipeline in its tracks.

To solve this, we pivoted to a Direct-to-Bucket architecture. I provisioned a Google Cloud Storage (GCS) bucket and configured our FastAPI backend to generate time-boxed signed URLs. This allowed the client to upload files directly to the cloud, completely bypassing the web server. Once uploaded, a new asynchronous pipeline pulls the data and processes it in memory-safe 500-record chunks.

However, when we moved to End-to-End (E2E) testing to verify this new architecture, we hit a wall.

Root Cause Analysis: 404 Dashboard Errors: As the terminal output shows, the frontend UI was attempting to fetch data from /api/companies/acme/provider_walkthrough and /api/companies/acme/records. These endpoints were returning 404 Not Found because they hadn't been fully scaffolded on the FastAPI side yet.

The 403 Upload Block: Concurrently, the backend strictly requires an X-User-Role HTTP header to authorize the signed URL generation. The React frontend was reading the user's admin role from the UI state but failing to attach it to the network request. Without that header, the backend defaulted to a read-only "Viewer" state and blocked the upload.

The Solution: Sprint 7 and Handshake Stabilization
To properly manage this technical debt, we scoped out an Agile sprint specifically dedicated to resolving these cross-system integration bugs. For Sprint 7, we created a 15-story-point Epic: End-to-End (E2E) Integration & Handshake Stabilization.

Here is how we divided the workload to patch the pipeline:

Auth & Header Parity: Collin is updating the React fetch requests to explicitly pass the UI state role into the X-User-Role header, clearing the backend’s security policies and resolving the 403 upload rejection.

Endpoint Alignment: I am scaffolding and wiring up the missing /records and /provider_walkthrough API routes so the client can successfully fetch and render the pipeline's output metrics without hitting the 404s we saw in the server logs.

E2E Dynamic Load Testing: Once the handshake is secure, I will conduct a dynamic analysis load test, pushing a massive dataset through the application. We will monitor the backend memory consumption logs to verify that the 500-record chunking pipeline permanently resolves the previous OOM crashes.

Graceful UI Error States: Instead of a generic failure message, the UI will now gracefully handle network drops and explicitly warn users if they lack the required editor permissions.

Software integration is rarely just about making the code work; it’s about making the systems communicate securely and gracefully. With this sprint board locked in, we have a clear path to a stable, production-ready upload pipeline. Check back next week for the results of our load test.

Scaling MedReachAI: Overcoming Cloud Limits with Signed URLs and Stream Processing

Scott Shoemaker — Tue, 22 Sep 2026 14:46:15 +0000

As my capstone partner Collin and I press forward into the final phases of MedReachAI, our focus has shifted from core feature development to production scalability and infrastructure hardening. This week, we kicked off Sprint 6, tackling one of the most critical engineering hurdles an enterprise data platform can face: safely ingesting massive, real-world datasets without crashing our server environment.

The Problem: Hitting the Cloud Infrastructure Wall
MedReachAI is designed to clean, scrub, and analyze massive Healthcare Provider (HCP) datasets—often containing hundreds of thousands of records spanning over 800 MB per file. During initial end-to-end testing, we attempted to upload a full-scale national dataset and immediately ran into a brick wall of infrastructure limits:

The Network Bottleneck: Google Cloud Run enforces a strict 32 MiB maximum limit on incoming HTTP requests. Any file payload sent directly through our API triggers an immediate HTTP 413 error before the data even reaches our backend logic.

Memory Exhaustion (OOM Crashes): Our FastAPI backend container is provisioned with 1 GiB of RAM. Attempting to parse an 800 MB CSV file synchronously into memory—while simultaneously running heavy spaCy and Presidio NLP models for PII detection and anomaly flagging—guarantees a fatal Out-of-Memory container crash.

The Solution: A Scalable Direct-to-Bucket Architecture
To solve these scaling challenges, we designed and planned a complete architectural overhaul under our Large File Handling Architecture epic. Instead of routing massive file payloads through our web server, we are decoupling file transport from backend processing using a three-tiered approach:

Backend Signed URL Generation: We are implementing a new FastAPI endpoint utilizing the google-cloud-storage SDK. When a user initiates an upload, the backend verifies their permissions and generates a secure, time-limited v4 Signed URL pointing directly to our Google Cloud Storage (GCS) bucket.

Direct-to-Client Bucket Upload: Collin is refactoring the React frontend upload workflow. Instead of sending the file body to our API, the client fetches the Signed URL and executes a direct PUT request to GCS, completely bypassing the 32 MiB Cloud Run request limit and enabling progress-tracked streaming over secure channels.

Asynchronous Stream Processing: Once the raw file safely lands in GCS, an event trigger notifies the backend to begin processing. Rather than loading the entire dataset into RAM, scrubbing_pipeline.py is being refactored to read the file as an asynchronous data stream, processing records in memory-safe chunks to keep our container well below its 1 GiB ceiling.

Looking Ahead
By shifting from direct HTTP payload transfers to a decoupled GCS storage and streaming model, MedReachAI will be fully equipped to handle enterprise-grade healthcare datasets smoothly and reliably.

Stay tuned as Collin and I execute these Sprint 6 tasks and bring our capstone architecture to its production-ready finish line!

Sprint 5: Breaking Local Constraints and Moving MedReachAI to the Cloud

Scott Shoemaker — Wed, 16 Sep 2026 15:16:12 +0000

The Problem: The Local Development Bottleneck
Up until this week, the MedReachAI application was strictly confined to local development environments. While this is fine for initial prototyping, it created a massive bottleneck for team collaboration. My project partner, Collin, was building the React frontend, while I was constructing the Python FastAPI backend. The immediate problem was connectivity and accessibility: the live Firebase frontend could not communicate with a backend running on localhost. Furthermore, simulating real-world latency, handling Cross-Origin Resource Sharing (CORS) security policies, and conducting end-to-end QA testing is impossible when an application isn't truly "live." We needed a scalable, cloud-hosted infrastructure that allowed independent, continuous deployment.

The Solution: Serverless Containerization
To solve this, Sprint 5 was entirely dedicated to cloud infrastructure migration. Instead of dealing with the overhead of heavy local container software, I opted for a lightweight, automated pipeline using Google Cloud Build and Cloud Run.

Here is how we resolved the infrastructure roadblocks:

Backend Containerization: I wrote a custom Dockerfile to package the FastAPI application, its dependencies (including heavier libraries like spaCy and pandas), and the server execution logic into a single, reproducible image.

Automated CI/CD Pipeline: By linking our GitHub repository directly to Google Cloud Build, any code pushed to the backend-dev branch now automatically triggers a cloud-side compilation. This offloads the hardware requirements from my local machine directly to Google's servers.

Serverless Deployment: Cloud Run takes that built container and hosts it on a public HTTPS endpoint. By allocating 1 GiB of memory, we ensured the Python environment has the resources it needs without risking Out-Of-Memory (OOM) crashes on startup.

_API Routing: _Finally, to bypass CORS limitations, we updated our firebase.json rewrite rules. Now, when the React frontend calls an API endpoint, Firebase securely proxies that request directly to the live Cloud Run instance.

The Result: A Fully Hosted Capstone
This architecture fundamentally changed our workflow. The backend API is now officially live on the web and fully integrated. We no longer have to spin up local terminals or worry about environment discrepancies.

Looking Ahead to QA
With the infrastructure hardened, the remainder of Sprint 5 is focused on optimizing database performance and aggressive stress testing.

Our upcoming backlog includes:

_Batched Writes & Indexing: _Optimizing bulk flag resolutions and implementing custom Firestore indexing to accelerate query speeds on the new cloud architecture.

Resilience Engineering: Designing asynchronous exponential backoff and retry logic to prevent data loss during network hiccups.

Stress Testing: Using tools like Locust and JMeter to evaluate concurrency limits, alongside rigorous vulnerability testing against malicious payloads and CSV injections.

The foundation is built; now it is time to see how much stress it can handle.

Taming Messy Healthcare Data: Entity Resolution and Overcoming React's "Invisible Shield"

Scott Shoemaker — Tue, 08 Sep 2026 12:58:14 +0000

Building a compliance tool like MedReachAI means confronting one of the healthcare industry's most expensive problems: messy data. Medical registries, pharmaceutical payment logs, and clinical databases are notoriously riddled with human typos, duplicate provider entries, and missing Protected Health Information (PHI).

When a compliance officer has to manually scrub a spreadsheet containing thousands of rows, the risk of human error is immense. Accidentally exporting unvalidated data can trigger severe regulatory liabilities. This week, my teammate Collin and I focused on automating this audit process by connecting our Python anomaly detection backend to our React frontend, creating a "Human-in-the-Loop" resolution dashboard.

The Problem: Fragmented Data and UI Disconnects
Our primary architectural goal this week was to implement automated entity resolution. We needed the system to recognize that two separate rows with missing data (e.g., a missing email in one, a missing phone number in another) actually belong to the same provider, rather than treating them as isolated errors.

While the Python backend was successfully clustering these duplicates using the provider's National Provider Identifier (NPI), we hit a significant technical roadblock on the React frontend.

We built a bottom table to display flagged records, intending for the user to click a provider's name and have the top dashboard instantly snap to that provider's detailed profile. However, nothing happened when we clicked the rows. The custom table components we were using were acting as an "invisible shield," silently swallowing the native onClick events. Furthermore, the derived arrays populating the table were stripping out the original database IDs, leaving React with no way to trace the click back to the source object.

The Solution: Native Spans and The "Golden Record"
To resolve the state management disconnect, we had to bypass the custom UI wrapper entirely.

First, we refactored the derived PII rows to explicitly preserve and pass the originalRecord object. Second, instead of attaching the click handler to the parent , we wrapped the provider's name cell in a native HTML . By attaching the onClick event directly to this native element and utilizing e.stopPropagation(), we forced the event through the custom table wrapper.

The moment this state connection clicked into place, the true power of our anomaly engine became visible.

Now, when you click on a provider—let's say, "Riley Flores"—the dashboard doesn't just show a single row of data. Because the system anchors its logic to the NPI, it instantly pulls in all associated duplicate records and clusters them under Riley’s primary profile. We call this the "Golden Record." The compliance officer gets full context in a single view, complete with dynamically calculated null-percentages for missing emails and phone numbers, and one-click controls to safely Anonymize or Remove the bad data.

What’s Next: Stress-Testing with CMS National Data
Up to this point, we have been validating our logic against a curated 60-row synthetic dataset. While it perfectly demonstrates our statistical anomaly detection—like catching a mathematically impossible 75,000 prescription volume—we need to prove this architecture scales.

Following some excellent feedback from our professor, our next major milestone is stress-testing the pipeline. I will be ingesting the official Centers for Medicare & Medicaid Services (CMS) "Doctors and Clinicians" national dataset. My goal is to map this massive, real-world file to our pipeline and ensure our Python engine can process, flag, and group the data without memory timeouts before cleanly passing it to the React frontend.

We have the logic locked in; now it is time to see how it handles the weight of national data.

Stepping into Beta: Explainable AI and Securing MedReachAI

Scott Shoemaker — Tue, 01 Sep 2026 15:49:21 +0000

As we officially transition from the Alpha phase of our Capstone into Beta development, the focus for MedReachAI is shifting from core functionality to real-world usability and enterprise-grade security.

Over the last few weeks, Collin and I successfully built a backend data intelligence pipeline capable of processing healthcare provider data, identifying statistical anomalies using machine learning, and verifying regulatory compliance against federal databases. But as we map out our next two-week sprint, we are preparing to tackle two massive hurdles: making our AI explainable to human stakeholders, and locking down the system architecture.

The Problem: The "Black Box" of AI
In our latest milestone review, we received fantastic feedback: the underlying architecture works, and the pipeline correctly flags bad data. However, there was a critical piece missing. Stakeholders don't just want to know that a record was flagged; they need to know why.

When an unsupervised machine learning model (like our IsolationForest) outputs a -1 to indicate an anomaly, it creates a "black box" scenario. For a healthcare data steward tasked with managing critical provider directories, seeing a mathematical variance score isn't actionable. If users don’t understand the AI’s decision-making process, they won't trust the platform.

The Solution: "Day in the Life" Walkthrough & Explainable AI
To solve this ambiguity, we are dedicating the first week of this sprint to building an interactive visualization that demystifies the AI pipeline. We plan to engineer a "Plain-Language Anomaly Explanation Generator." Instead of just returning a variance score, our goal is for the backend to dynamically translate the mathematical anomaly into human-readable context (e.g., "Flagged because billing volume is 5x the Cardiology average").

To build and test this feature, I will be writing a Python script to generate a highly curated synthetic dataset. We intend to intentionally inject four distinct "bad data" personas into this batch to test every layer of our pipeline:

The Statistical Anomaly: A provider with massive prescription volumes to test the machine learning model and trigger the future plain-text generator.

The NPI Failure: A deactivated National Provider Identifier to ensure our CMS validation flag catches it.

The Financial Conflict: A provider exceeding the $100 Sunshine Act limit to trigger our local SQLite compliance database.

The Duplicate: An identical record to trigger our Pandas soft-delete flag.

Once this dataset is generating properly, Collin will build out a "Record Detail" UI in React. This will allow us to step through these exact records and visually demonstrate the pipeline's logic to our stakeholders in real-time.

The Problem: Securing Healthcare Infrastructure
The second major hurdle for our Beta release is security. Up to this point, our local environment allowed open communication between our React frontend and FastAPI backend. In a production healthcare environment, leaving endpoints unauthenticated is an absolute non-starter. We need strict access controls to ensure that only authorized personnel can view or manipulate data, and that data from one organization cannot bleed into another.

The Solution: JWT Middleware and RBAC
For the second half of this sprint, my focus will shift entirely to Phase C architecture: Security and Authentication.

I plan to implement a robust Firebase Auth JSON Web Token (JWT) Middleware layer on the FastAPI backend. Every single API request will require a verified token before the server even looks at the data. Once the perimeter is secure, we will roll out Role-Based Access Control (RBAC).

Admins will have full access to team management and data scrubbing.

Editors can review and flag data.

Viewers will be strictly read-only.

Coupled with upcoming multi-tenant Firestore security rules, this will ensure our platform is isolated, secure, and ready for a true Beta deployment.

Moving Forward
This two-week hybrid sprint is designed to perfectly bridge the gap between demonstrating immediate value to our stakeholders and building the invisible, architectural security foundation required for a production-ready application. I am looking forward to seeing this "Day in the Life" walkthrough come to life and reporting back on how the plain-language generator handles the synthetic data next week.

MedReachAI: Bringing Data Intelligence to Healthcare Provider Management

Scott Shoemaker — Thu, 27 Aug 2026 21:02:52 +0000

The Messy Reality of Healthcare Data (The Problem)
When people hear the phrase "healthcare data," they immediately think of patient records—blood pressure, clinical notes, and heart rates. But there is an entirely different side to the industry that is just as critical, and often just as messy: Provider Data.

Healthcare organizations, compliance officers, and medical sales teams rely on massive datasets of physician credentials, practice locations, and financial relationships. However, this data is notoriously unstructured. It suffers from a few major problems:

Corruption & Duplication: Provider databases are often cobbled together through web-scraping or manual entry. This leads to duplicate profiles, mismatched specialties, and corrupted National Provider Identifiers (NPIs).

Hidden Security Risks: Free-text fields (like notes or contact arrays) frequently harbor Protected Health Information (PHI) or Personally Identifiable Information (PII), creating massive compliance liabilities.

Financial Blind Spots: Without a clean, verified NPI, it is impossible to track a physician's financial relationships under the federal Sunshine Act.

The industry needs a way to automatically ingest, sanitize, and secure this provider data before it ever hits a production database.

(Caption: Raw healthcare provider datasets are often riddled with structural errors and hidden compliance risks.)

Enter MedReachAI (The Solution)
Over the past few months, my capstone partner, Collin, and I have been building MedReachAI: an automated, full-stack data intelligence platform designed specifically to sanitize and enrich healthcare provider profiles.

Rather than relying on humans to manually comb through spreadsheets, we engineered a backend pipeline using Python, FastAPI, and machine learning to do the heavy lifting.

Here is a look at the core features driving the MedReachAI backend:

Unsupervised Anomaly Detection
To catch corrupted data, I engineered a machine learning model using Scikit-Learn’s IsolationForest. Because provider data changes dynamically, we don't rely on a static training set. Instead, the algorithm evaluates each new upload batch in real-time, calculates the mathematical distance between provider profiles, and automatically isolates the top 5% most irregular records (such as geographic mismatches or structural scraping errors).
Automated PII & PHI Redaction
To handle security, I integrated the Microsoft Presidio AnalyzerEngine. I built custom pattern recognizers that scan unstructured text to detect Medical Record Numbers (MRNs) alongside standard identifiers like SSNs and emails. Once detected, our downstream Anonymizer service actively scrubs the data, replacing the sensitive text with explicit tags like .

(Caption: The MedReachAI backend utilizes custom Python services to flag statistical anomalies and redact sensitive information in real-time.)

Non-Destructive Data Deduplication
To ensure our analytics remain accurate, I built a custom Pandas algorithm to identify exact and partial duplicate provider profiles. Instead of deleting these records and risking data loss, the system performs a "soft delete" by appending a boolean flag to the database row, allowing the data to be recovered if needed.
The Human-in-the-Loop Dashboard
While the backend handles the automation, Collin built a brilliant React-based UI that gives the user final oversight. The MedReachAI dashboard features a suite of review tabs that allow end-users to manually accept or reject the anomaly, duplication, and PII flags generated by the backend.

(Caption: The MedReachAI dashboard provides end-users with real-time visibility into the health and cleanliness of their provider datasets.)

What’s Next?
With the core data-cleaning pipeline fully operational, Phase C of our development will focus on external enrichment. I will be building asynchronous clients to connect our clean local records directly to the official CMS NPI Registry and the CMS Open Payments database, turning sparse, sanitized records into rich, compliant provider profiles.

Building MedReachAI has been an incredible exercise in bridging the gap between raw data engineering and user-facing application design. Stay tuned as we wrap up our final features!