DEV Community: Gokul Suresh

Testing AWS-integrated apps for free: a local setup with Floci and ngrok

Gokul Suresh — Fri, 17 Jul 2026 07:08:11 +0000

Build, test, and share AWS-powered applications without an AWS account or deployment.

The problem I kept running into

If your app talks to AWS — uploading files to S3, queuing jobs through SQS, running functions on Lambda — development normally forces an uncomfortable choice.

Option one: write separate "fake" code paths just for local development (if (isDev) return fakeData). This means the code path you're actually testing every day is not the code path that runs in production. Bugs in your real AWS integration logic slip through, because you never actually exercised it locally.

Option two: use a real AWS account for development too. This works, but it costs money for every test run, requires every teammate to have their own AWS credentials, and means someone has to manage IAM permissions just so people can build features.

Neither is great. There's a third option that solved this for me completely, and it costs nothing: run a local AWS look-alike on your own laptop, and use a tunneling tool to let others access what's running there. This post walks through exactly how, and why each piece exists.

The two tools, and what they actually do

Floci — a local AWS emulator

Floci is an open-source program that emulates AWS's API surface on your own machine. It listens on localhost:4566 and responds to the same requests real AWS would — create a bucket, upload a file, invoke a function — using the exact same request format Amazon defined.

The key detail: your application code doesn't know the difference. If you're using the official AWS SDK (@aws-sdk/client-s3, boto3, whatever your language's equivalent is), that code has no idea whether it's really talking to Amazon or to a program pretending to be Amazon. It just sends requests to whatever address it's configured with.

For some services, Floci goes further than a shallow mock — it spins up real backing containers. Point it at "RDS" and it actually runs Postgres. Point it at "ElastiCache" and it actually runs Redis. So you're not just getting plausible-looking fake responses; for the stateful services, you're getting the real thing, just running locally instead of in Amazon's data centers.

ngrok — a tunnel to your laptop

Once your app is running locally, it's still only reachable from your own machine — localhost means "this same computer," and your home or office router blocks incoming connections from strangers by default (a security feature, not a bug).

ngrok solves this with a trick worth understanding, because it explains why no firewall configuration is ever needed: your laptop opens an outbound connection to ngrok's servers (always allowed, since outbound traffic isn't blocked). That connection stays open, like a phone call that never hangs up. ngrok then hands you a public URL. When someone visits it, ngrok's servers receive the request and relay it back down the open connection to your laptop, which replies, and the reply relays back out the same way.

Nobody ever connects to your laptop directly. They connect to ngrok, and ngrok relays through the tunnel your laptop itself opened.

Request flow diagram

Simplified down to just the request/response round trip, here's what happens every time someone opens your ngrok link:

Browser
   │  1. GET https://xyz.ngrok.app
   ▼
ngrok edge server (public IP)
   │  2. forwarded down the open tunnel
   ▼
ngrok agent (your laptop)
   │  3. localhost:3000
   ▼
Your app handles the request
   │  4. response travels back up the same path
   ▼
Browser renders the page

The full picture

Visitor's browser
        │
ngrok's public server (has a real public IP)
        │
Open tunnel (started BY your laptop, stays open)
        │
ngrok agent running on your laptop
        │
Your app (e.g. localhost:3000)
        │
AWS SDK calls -> Floci (localhost:4566)
        │
Real Docker containers for stateful services
        │
Data persisted to a folder on your own disk

Three independent layers: your application code (unchanged), an emulation layer intercepting AWS calls, and an access layer exposing the whole thing publicly if you want to share it.

Architecture diagram

A cleaner, boxed view of the same setup — useful as a quick visual reference for the stack running on your machine:

                Internet
                    │
                    ▼
          ngrok Public URL
                    │
        Secure Tunnel (HTTPS)
                    │
                    ▼
┌──────────────────────────┐
│      Local Machine        │
│                           │
│  React / Next.js Frontend │
│            │              │
│  Express / Nest Backend   │
│            │              │
│         AWS SDK           │
│            │              │
│          Floci            │
│            │              │
│  S3 • DynamoDB • SQS      │
│  Lambda • Secrets Manager │
└──────────────────────────┘

Development workflow

Laid out as a loop, this is the day-to-day cycle the setup is built around:

Write Feature
      │
      ▼
Run Application Locally
      │
      ▼
Floci emulates AWS
      │
      ▼
Test Uploads
      │
      ▼
Share using ngrok
      │
      ▼
Receive Feedback
      │
      ▼
Deploy to Render

Setting it up, step by step

1. Install Docker and ngrok

Floci runs inside a Docker container, so Docker needs to be installed and running first. ngrok is a separate download from ngrok.com.

Verify Docker is working:

docker info

ngrok config add-authtoken <your-token>

2. Write a docker-compose.yml

This file tells Docker which image to run and how to configure it:

services:
  floci:
    image: floci/floci:1.5.11
    container_name: local_aws_cloud
    ports:
      - "4566:4566"
    environment:
      - FLOCI_STORAGE_MODE=persistent
      - SERVICES=s3,dynamodb,sqs,lambda,rds,secretsmanager
    volumes:
      - ./data:/app/data
      - /var/run/docker.sock:/var/run/docker.sock

A few details worth explaining rather than just copying:

Pin a specific version tag (1.5.11, not :latest). If everyone on a team pulls :latest, people end up on different versions with different env var names, and setups silently drift apart.
./data:/app/data is a bind mount — it maps a folder on your real hard disk to a folder inside the container. This is what makes storage survive restarts; without it, everything Floci stores would vanish the moment the container stopped.
The Docker socket mount (/var/run/docker.sock) is what lets Floci spin up real containers for services like RDS and Lambda, instead of faking them. It's also a real security boundary worth knowing about — more on that below.

3. Start it and confirm it's healthy

docker compose up -d
curl http://localhost:4566/_localstack/health

The health check returns a JSON list of every AWS service Floci is currently emulating and whether each is running. If this doesn't come back, nothing downstream will work, so it's worth checking before moving on.

4. Point your app's AWS SDK at it

Set these before running your app:

AWS_ENDPOINT_URL=http://localhost:4566
AWS_ACCESS_KEY_ID=111111111111
AWS_SECRET_ACCESS_KEY=test
AWS_DEFAULT_REGION=us-east-1

Floci accepts any credential value — it doesn't validate them the way real AWS does. The one exception: a 12-digit numeric access key gets treated as an account identifier, useful if multiple people share one Floci instance and want isolated resources. Anything else falls into a shared default account.

If your code constructs its own S3Client (or equivalent), it needs to actually read this endpoint variable — the SDK doesn't automatically know to redirect just because the variable exists, unless your code passes it through explicitly:

const s3 = new S3Client({
  credentials: { accessKeyId: process.env.AWS_ACCESS_KEY_ID, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY },
  region: process.env.AWS_REGION || 'us-east-1',
  endpoint: process.env.AWS_ENDPOINT_URL || undefined,
  forcePathStyle: !!process.env.AWS_ENDPOINT_URL,
});

The forcePathStyle flag matters and is easy to miss. Real AWS S3 prefers requests shaped like bucket-name.s3.amazonaws.com. Local emulators like Floci expect the older path style instead: localhost:4566/bucket-name. Without this flag, requests can fail even with the endpoint correctly configured — a genuinely confusing failure mode if you don't know to look for it.

5. Create a bucket

S3 requires a bucket to exist before anything can be uploaded into it — this rule holds on Floci exactly as it does on real AWS:

aws --endpoint-url http://localhost:4566 s3 mb s3://your-app-files

The name has to match whatever your app's config expects, or uploads will fail with a "bucket does not exist" error that has nothing to do with your actual code.

6. Prove it end to end

Run your app, trigger a real upload through the actual UI, then check that the file landed:

aws --endpoint-url http://localhost:4566 s3 ls s3://your-app-files/ --recursive

When I did this on a real project, a genuine uploaded file showed up immediately — same upload code path production will use, running entirely for free on a laptop. Worth noting: only the raw file bytes live here. Structured data — filenames, uploader, timestamps — typically stays in your actual database (Mongo, Postgres, whatever you're using), completely separate from Floci. Databases hold records and pointers; S3-style storage holds the heavy binary data. That split doesn't change for local dev — you've just relocated where the storage physically lives.

7. Share it, if you want to

ngrok http 3000

Tunnel only your app's port. Never tunnel the Floci port (4566) directly. If someone reaches that port — through a second tunnel, a misconfigured proxy, or a CORS mistake — they get unauthenticated access to every emulated AWS service on your laptop, including the ones backed by real Docker containers with real host access.

Security considerations worth actually reading

The Docker socket mount is a real trust boundary. Floci having access to /var/run/docker.sock means a compromised emulator has host Docker control. Don't run this setup with that mount on a machine you'd consider sensitive.
Treat anything uploaded through the tunnel as landing in plaintext on your disk. Don't use this setup for real customer data — it's a development and testing tool by design, not a secure storage system.
For anything beyond a same-day test, use ngrok's access controls (basic auth or an IP allowlist) in front of the tunnel. Free tier doesn't include this, so a bare link should be treated as something anyone with it can open.
Your app's own authentication still applies. A tunnel just gets someone to your login screen — it doesn't bypass whatever session/auth logic your app already enforces.

ngrok's free tier, in actual numbers

As of ngrok's 2026 pricing, the free plan gives you: up to 3 concurrent endpoints, 1GB of data transfer per month, and 20,000 HTTP requests per month — not per day, per month, shared across everyone who visits. There's no hard session timeout — an endpoint can stay online continuously as a background service. First-time visitors do see an interstitial warning page before reaching your app (a 7-day cookie skips it after their first visit), and the public URL rotates on every restart unless you're on a paid plan with a reserved domain.

One clarification worth making: "3 concurrent endpoints" means 3 separate tunnels you can run at once — it has nothing to do with how many people can visit one tunnel's link simultaneously. Any number of people can open the same shared link at the same time; the actual ceiling is the monthly bandwidth and request totals, not concurrent visitor count.

Where this fits, and where it doesn't

This setup is not a replacement for real hosting. It's genuinely good for exactly two things: catching integration bugs against AWS-shaped services before they ever reach a real AWS bill, and sharing a live, working demo with someone in under a minute without deploying anything.

It's genuinely not good for anything resembling production traffic. Your laptop has to stay on, the free tier caps are real, and there's no uptime guarantee if your machine sleeps or your network drops.

For actual hosting — real users, real uptime — platforms like Vercel or Render are the right tool, and they solve a different problem entirely. One detail worth knowing if your app uses WebSockets (Socket.io, for example): Vercel's serverless functions don't support long-lived persistent connections, so a platform like Render, which runs actual persistent servers, is often the better fit for anything beyond simple HTTP APIs.

Floci + ngrok vs. Render / Vercel

Side by side, the two solve different problems — here's the quick version:

Feature	Floci + ngrok	Render / Vercel
Purpose	Development	Production
AWS Cost	Free	Real AWS
Deploy Required	No	Yes
Share Demo	Yes	Yes
Always Online	No	Yes
Laptop Required	Yes	No

The realistic workflow is to use both, at different stages: local Floci + ngrok while actively building and testing, and a real host once a feature is stable enough for real users to depend on it.

Technical Documentation: Integrating Eraser.io via Model Context Protocol (MCP)

Gokul Suresh — Mon, 13 Jul 2026 18:48:13 +0000

This document provides a comprehensive overview of configuring and utilizing the Eraser.io Model Context Protocol (MCP) server across various AI clients. Integrating Eraser via MCP transforms your static system architecture diagrams and markdown files from standard developer documentation into active context for your AI workflows.

1. Architectural Overview: How Eraser MCP Works

The Model Context Protocol (MCP) is an open standard designed to securely bridge the gap between LLMs (Large Language Models) and external data sources. Instead of forcing developers to constantly copy-paste snippets or feed heavy text files into context windows, Eraser acts as an MCP Host/Server.

┌─────────────────┐        MCP Request        ┌───────────────────┐
│  AI Client /    │ ────────────────────────> │ Eraser MCP Server │
│  Coding Agent   │ <──────────────────────── │ (app.eraser.io)   │
└─────────────────┘       Context JSON        └───────────────────┘
         │                                              │
         ▼                                              ▼
 Reads Local Codebase                         Fetches Architecture,
 & Project Files                              Flowcharts & Tech Docs

When connected, your AI assistant can run native tools exposed by Eraser to programmatically query, read, and understand systemic engineering logic, creating an environment where code generation aligns perfectly with pre-defined architecture boundaries.

The Problem of Context Rot

Traditional engineering workflows suffer from documentation decay. Developers update code, but visual diagrams in external tools become outdated. By exposing live canvas objects as machine-readable state via an MCP server, your AI assistants gain visibility into:

Current boundary definitions and microservice scopes.
Up-to-date data schemas and entity relationships.
Active security rules, encryption boundaries, and compliance mandates.

2. Configuration Settings Breakdown

Based on the interface shown below, the core connection engine relies on a standardized HTTP transport protocol. Below is the primary endpoint layer used across integrations:

MCP Settings Path: Accessible via Personal Settings → MCP in the Eraser dashboard.
Base Connection URL: https://app.eraser.io/api/mcp
Transport Mechanism: HTTP
Authentication Mechanism: Managed via Bearer tokens generated natively by the specific integration flow, or through the API Tokens tab under Team Settings.

3. Client-Specific Integration Playbook

Eraser natively exposes integration vectors for six core AI developer tools. Here is how to configure and utilize each interface.

1. Claude Code (Anthropic's Agentic CLI)

Claude Code executes directly within your terminal, acting as an interactive engineering agent capable of executing local commands, managing state, and writing software.

Setup Implementation: Run the following command directly in your project terminal:

claude mcp add --transport http eraser https://app.eraser.io/api/mcp

Operational Capability: Once initialized, Claude Code queries the endpoint on demand. If you ask it to build a new microservice route, it checks your active canvas first to preserve compliance, isolation, and formatting criteria.

Primary Use Case: Agentic terminal operations where the model creates files, runs build scripts, and refactors large directory structures while referencing systemic documentation.

2. Claude.ai (Web Interface)

For high-level system architectural debates, product feature scoping, and design review processes conducted outside the terminal environment.

Setup Implementation: Select the Claude.ai client block in Eraser and follow the OAuth authorization prompt to allow the official connector access to your workspace.

Operational Capability: Allows you to query the web model directly using conversational requests like: "Review this database schema from my Eraser workspace against our tenant isolation policy."

Primary Use Case: Early-stage planning, architectural decision records (ADRs), and non-developer technical strategy reviews.

3. OpenAI Codex

Ideal for legacy orchestration setups and developers using custom workflows built around OpenAI's foundational code models.

Setup Implementation: Point your environment's manual MCP JSON configuration block to target the Eraser API transport layer using your secure API token header.

Operational Capability: Supplies raw code completion setups with explicit definitions of complex dependencies, network edges, and microservice topologies.

Primary Use Case: Automated CI/CD pipeline script generation and custom internal AI tooling configurations.

4. VS Code (Visual Studio Code)

Integrates deep architectural context inside the industry's standard code editor.

Setup Implementation: Configured via the workspace or global settings file. You append the Eraser server path directly into the JSON configuration under the server list.

Operational Capability: Empowers inline chat mechanisms, prompt sidebars, and local inline edits to respect data flow diagrams and existing module logic.

Primary Use Case: Day-to-day feature development where developers need immediate, inline feedback based on established architectural patterns.

5. Cursor

Cursor is designed from the ground up for deep indexing of local repositories. By pairing it with Eraser, it indexes the architectural mapping alongside the codebase.

Setup Implementation: Navigate to Cursor Settings → Features → MCP. Click Add New MCP Server, choose http as the type, and input https://app.eraser.io/api/mcp along with your validation token.

Operational Capability: Enables deep contextual symbol lookups and multi-file editing that respects visual boundaries, sequence diagrams, and module interactions.

Primary Use Case: Complex code generation tasks spanning multiple files and services where strict architectural adherence is required.

6. GitHub Copilot

Brings systemic awareness to the default AI pair-programmer natively integrated into the GitHub ecosystem.

Setup Implementation: Configured via the Copilot Chat extensions hub or custom JSON manifests inside enterprise development environments using the HTTP endpoint.

Operational Capability: Cuts down on generic boilerplate code suggestions by forcing Copilot's autocompletion logic to operate within the specific guidelines defined inside your team's technical documentation.

Primary Use Case: Auto-completing daily function bodies and boilerplate with exact type definitions and error-handling routines specified in project documents.

4. Operational Comparison Matrix

AI Client	Interface	Primary Connection Method	Key Architectural Benefit
Claude Code	Terminal CLI	`claude mcp add` CLI command	Full agentic filesystem access guided by system design.
Claude.ai	Web UI	Direct OAuth / Connector	High-level system design debates and ADR formulation.
OpenAI Codex	Custom / API	Endpoint Configuration JSON	Automated pipeline tooling and script generation.
VS Code	IDE	Extension / JSON Settings	Direct IDE context for standard development loops.
Cursor	Native AI IDE	MCP Settings UI Panel	Multi-file generation adhering to visual flowcharts.
GitHub Copilot	IDE Plugin	Copilot Extension Manifest	In-line completions aligned with documented schema specs.

5. Engineering Best Practices for MCP-First Documentation

To get the most out of Eraser's MCP server, documentation should be structured with the AI's perspective in mind:

Keep Text and Diagrams Together: Eraser allows anchoring raw Markdown notes directly beneath a visual diagram. Use this to describe architectural decisions, edge cases, or security parameters that a diagram alone cannot convey.
Establish a Single Source of Truth: Replace disconnected external files with active workspace canvases. Keep the architecture dynamic and centralized so the AI model never pulls from an outdated specification.
Semantic Naming Conventions: Give clear, explicit names to your files, objects, and modules (for example, Level 1 – System Overview or Authentication Flow). Clean naming conventions make it easier for the AI to locate, parse, and inject context during code generation.
Define Clear Boundaries in Markdown: Annotate diagrams with clear text headings specifying module boundaries, supported protocol versions, database constraints, and network ports. AI models read these explicit directives to generate accurate API signatures.

The Silent RAG Killer: How Noisy Document Layouts Can Burn 70,000 Tokens Before Your First Prompt

Gokul Suresh — Tue, 07 Jul 2026 06:00:51 +0000

Hey Dev Community! I originally published a version of this breakdown over on my Medium, but I wanted to share the full technical breakdown directly with the engineers here. If you're building custom AI pipelines, this one is for you.

Every engineer building with Large Language Models (LLMs) eventually hits a hidden wall. You write a pristine system prompt, optimize your retrieval-augmented generation (RAG) code, pipeline your vector database, and push to staging. Everything looks brilliant.

Then you open your usage dashboard and look at the actual token count.

I recently hit this wall while researching how to integrate a geolocation service into our company’s core project. Like anyone trying to parse mountains of engineering guidelines, API references, and spatial data manuals, I started gathering raw documents — PDFs, Word docs, and web pages — and feeding them into Claude to fast-track my implementation research.

Out of curiosity, I paused to look closely at the underlying token consumption. What I discovered was a massive wake-up call for anyone building real-world AI applications.

The Math Behind Token Bloat
When we interact with text via an LLM’s user interface, we don’t think about the invisible baggage tucked inside our documents. But behind the scenes, raw files are packed with semantic noise: XML tags inside .docx architectures, absolute coordinates and layout elements inside PDFs, CSS styling from web scrapes, and dense formatting metadata.

When I checked the numbers for my unstructured research files, the breakdown was brutal:

Average Per-Page Cost: ~3,000 tokens per page of raw, unstructured layout text.
The Accumulation: A humble bundle of 20 reference pages consumed 70,000+ tokens.
[20 Pages of Raw Document] ──► Includes layout tags, margins, XML, and styling ──► 70,000+ Tokens Ingested
│
Cost incurred BEFORE ◄────────┘
your first question!
This isn’t just a financial drain; it’s an application-performance killer. Dropping a massive chunk of metadata noise into an LLM’s context window forces the attention mechanism to dilute its processing power. It has to look through thousands of tokens of styling code just to find the actual paragraph it needs to answer your question. The result? Higher latency, hallucinated references, and degraded reasoning quality.

Enter MarkItDown: Upstream Data Cleansing
To fix this, I needed a solution that would clean the data layer before it reached the model. I found exactly that in a highly trending open-source tool from Microsoft: MarkItDown (which recently crossed an impressive 163,000+ stars on GitHub).

Download the Medium app
MarkItDown is a lightweight Python utility designed to solve one specific problem: take messy file formats — including PDFs, Word, Excel, PowerPoint, ZIP files, and even YouTube video transcriptions — and convert them into pure, clean Markdown.

📁 Raw Files (.pdf, .docx, .xlsx, .pptx)
│
▼
⚡ [ Microsoft MarkItDown ]
│
▼
📝 Structured, Stripped Markdown Text (Ready for LLM)
Why Markdown is the Ultimate AI Ingestion Format
Token Minimalism: Markdown strips out layout structures, hidden XML trees, and presentation metadata, leaving only the essential content. It slashes your context window footprint dramatically.
Native Model Alignment: Frontier models like Claude and GPT-4o don’t just understand Markdown — they natively output it without being asked. They were trained on vast oceans of pure Markdown code. When you pass an LLM data in its “native tongue,” its retrieval and synthesis logic perform significantly better.
Preserved Semantics: Unlike raw text-stripping tools that smash everything into an unreadable block of text, MarkItDown keeps headings as #, lists as *, and complex Excel spreadsheets as perfectly aligned Markdown tables. The structural meaning remains completely intact.
The Ultimate Power-Up: Model Context Protocol (MCP)
What makes MarkItDown uniquely built for modern AI workflows is its native support for the Model Context Protocol (MCP).

Using an implementation like markitdown-mcp, you don't even need to write script pipelines to handle your local files. You can plug the server straight into your Claude Desktop application configuration file:

JSON

{
"mcpServers": {
"markitdown": {
"command": "markitdown-mcp",
"args": []
}
}
}
Once configured, your AI assistant can directly call the file converter utility on demand, transforming documents on your machine into token-efficient Markdown right inside your chat session.

Where Does Google’s NotebookLM Fit In?
While mapping out this approach, another tool naturally crossed my mind: Google’s NotebookLM. It’s an incredible app, and it’s worth distinguishing when to use which.

Use NotebookLM if: You need an out-of-the-box, consumer-facing research environment. It excels at auto-ingesting documents, synthesizing complex notebooks, and generating audio overviews seamlessly.
Use MarkItDown if: You are an engineer building a custom production pipeline. If you need granular programmatic control, data pipeline automation via Python, or a local utility that fits directly into your proprietary codebase, cleaning your data upstream with MarkItDown is the professional choice.
Takeaway for AI Developers
Garbage in, garbage out. Half the challenge of building dependable RAG pipelines or engineering workflows isn’t selecting the flashiest model; it’s mastering your data ingestion layer.

Before you drop another unoptimized PDF or Word doc into your context window, think about the 70,000 tokens you are burning on layout noise. Clean your text, leverage Markdown, and optimize upstream. Your budget — and your model’s accuracy — will thank you.

Have you experimented with document pre-processing tools for LLM workflows? Let’s talk about your data ingestion strategies in the comments below!

🛠️ Open Source Project: https://github.com/microsoft/markitdown

Let's Connect! 🚀
I am an engineer heavily focused on optimizing AI pipelines, data ingestion frameworks, and RAG systems. If your team is building something in the Generative AI space or if you're looking for an engineer who treats token optimization as a core metric, let's chat!

🔗 Connect with me on LinkedIn = https://www.linkedin.com/in/gokul-dev1/

📁 Check out my open-source work on GitHub = https://github.com/Gokulsuresh1918