DEV Community: Sanskar Kharya

The Guide That Sees for You: Building an On-Device Navigation Companion in the Browser

Sanskar Kharya — Thu, 27 Aug 2026 07:14:22 +0000

I built a browser app that turns a live camera feed into 3D sound, so a visually impaired person can hear where things are. It runs entirely on-device — no cloud, no network. What exists today is a working prototype in a web browser; the larger goal is a native phone app running on dedicated AI silicon.

This post walks through the idea, what actually works right now, and the honest gap between the two.

Live demo: https://saartheye-ai.vercel.app
Code: https://github.com/MaybeSomeone-arc18/saartheye-ai

Why a phone in the cloud is too slow to keep you safe

A safety warning that arrives a second late is useless. Most AI vision apps send a camera frame to a server, run the model there, and send the answer back. That round trip takes roughly 1.2–3.5 seconds. A person walking at a normal pace covers nearly five feet in that window — long enough to have already hit the obstacle the app is warning about.

There are three problems with the cloud approach for real-time navigation:

Latency. The round trip is slower than the thing it's meant to prevent.
Dead zones. Elevators, basements, subways, parking garages — the places you most need help are the places with no signal. A cloud app stops working the moment the connection drops.
Privacy. A navigation aid you wear all day streams a 24/7 video of your entire life — your home, your workplace, the strangers around you — to someone else's servers.

Doing everything on the device sidesteps all three at once: no round trip, no signal required, and camera frames never leave the phone. That premise is the whole reason this project exists.

What I actually built

Everything happens inside the web page — there's no backend server at all. The prototype opens your camera, spots objects in the video, and plays them back to you as spatial audio in real time.

Vision: a YOLO-based object detector running through TensorFlow.js, so inference happens on your machine, not a server.
Audio: the browser's Web Audio API, which can place a sound at a point in 3D space around your head.
Interface: a React layer that draws the detection boxes and mode controls on screen.

Why the browser first? It was the fastest way to prove the core idea is real. It let me test the full loop — camera → detection → sound — with zero install, on any laptop or phone, in a few days of building. It's the proof of concept, not the finished product.

From pixels to sound

1. Finding things: the detector

Each video frame goes through an object detector that returns a list of what it sees and where. For every object, the model gives back a label, a confidence score, and a box (its position and size on screen). That box is the single most important piece of data — everything downstream is built from it.

interface DetectedObject {
  class: string;          // e.g. 'person', 'car', 'chair'
  score: number;          // confidence, 0.0 to 1.0
  bbox: [
    number,               // x (top-left)
    number,               // y (top-left)
    number,               // width
    number                // height
  ];
  spatialVelocity?: number; // how fast it's approaching (see below)
}

2. Turning a box into a sound you can locate

The trick is echolocation: map where an object sits in the frame to where its sound sits around your head. Two simple mappings do most of the work, and because they're just arithmetic on the box, they're effectively instant.

What you hear	Comes from	So that…
Left / right (stereo pan)	The box's horizontal position	An object on your left sounds on your left
Pitch (high / low)	The box's size on screen	A bigger, closer object sounds lower and nearer
Urgency of the pulse	How fast the object is approaching	A fast approach sounds more insistent

Under the hood this uses the Web Audio API's PannerNode, which is designed to position sound in 3D. The result: you don't get a robotic voice saying "chair, two o'clock" — you just hear the chair, roughly where it is.

3. Telling a threat from a table: velocity vectoring

Distance alone is a bad alarm. A chair one meter away isn't dangerous; a cyclist one meter away and closing is. The distinction isn't how close something is, but whether it's getting closer — and how fast.

The prototype keeps a short memory of each object's box across recent frames and watches whether that box is growing. A box that swells quickly frame-over-frame means something is rushing toward you, so it escalates to a sharp, panned warning tone. A box that holds steady is treated as furniture or a standing person and fades into a soft, occasional ping.

4. Reading the room: contextual modes

An aid that screams at everything gets switched off within a minute. Avoiding that "false-positive fatigue" is a feature, not an afterthought.

Outdoor / navigation: high sensitivity to anything moving or in your path — bikes, poles, stairs, people.
Social / conversation: a person standing still in front of you is a friend, not a hazard, so their alarm is suppressed to a gentle ambient ping. Only a sudden approach breaks through.

The honest gap: prototype vs. blueprint

The architecture I designed is more ambitious than the thing I've shipped, and it's worth being clear about which is which. The browser prototype proves the concept end to end. The native version is the target, not a claim about today.

	Prototype (today)	Blueprint (the goal)
Runs on	Web browser tab	Native Android app
Detection	YOLO via TensorFlow.js	Quantized YOLOv12 on the phone's neural chip
Reasoning	Bounding-box math in JavaScript	On-device small language model (Phi-3.5)
Feedback	3D spatial audio	Spatial audio + haptic vibration
Latency	Fast, browser-limited (unmeasured)	Sub-15ms target on dedicated silicon
Offline	Yes, once loaded	Yes, full airplane mode by design

The prototype already answers the risky question: can bounding boxes become useful, locatable sound in real time? Yes. The remaining work — native memory handling, offloading the model to a dedicated neural processor, adding haptics — is a systems-engineering climb, not an unknown. The concept is de-risked; the performance is the next mountain.

Where it goes next

Port the pipeline to native so camera frames flow straight into the model without the browser's overhead.
Offload detection to the phone's neural chip to chase the sub-15ms target and free the main processor.
Add haptics so a warning you can feel backs up the one you hear, for loud environments.
Test with actual users, because the real measure isn't latency — it's whether someone trusts it enough to leave the cane at home.

You can build the hard, scary part of an idea in a browser in a weekend and learn whether it's worth pursuing. This started as a question about whether sound could stand in for sight, closely enough to be safe. The prototype says it's possible. The rest is engineering.

The name comes from the Sanskrit Saarthi — a charioteer, the one who guides — joined with Eye.

Why I Don't Put Tenant-Specific Validation in My FastAPI Endpoints

Sanskar Kharya — Mon, 10 Aug 2026 10:09:21 +0000

Multi-tenant applications have a way of turning simple endpoints into surprisingly complicated pieces of code.

At first, tenant-specific validation seems harmless.

You have two tenants with different limits, so you write something like:

if tenant == "acme":
    max_quantity = 100
elif tenant == "globex":
    max_quantity = 10

It works.

The problem isn't the first if.

The problem is what happens after there are ten more rules.

When the endpoint starts knowing too much

I prefer an endpoint to answer one main question:

What should the application do with this request?

I don't want the route handler to also be responsible for figuring out:

which tenant made the request
whether that tenant is active
what limits the tenant has
which validation rules apply
how those validation failures should be returned

None of those things are really the endpoint's job.

They are context and validation concerns.

Once those responsibilities start accumulating inside a route, even a small endpoint can become difficult to reason about.

FastAPI dependencies are a natural boundary

FastAPI's dependency injection provides a useful place to resolve tenant information.

For example, a request can contain:

X-Tenant-ID: globex

A dependency can turn that header into a strongly typed TenantConfig.

Now the rest of the request-processing pipeline doesn't need to repeatedly parse the header or look up the tenant.

But that still leaves an interesting problem.

The request body doesn't contain the information we need for every validation rule.

Some validation needs context

Consider a simple Pydantic model:

class OrderCreate(BaseModel):
    product_id: str = Field(..., min_length=1)
    quantity: int = Field(..., gt=0)

The model can determine that quantity=0 is invalid without knowing anything about the tenant.

But suppose:

acme   → maximum quantity = 100
globex → maximum quantity = 10

Now the validity of the same request depends on information outside the request body.

That's where Pydantic v2's validation context becomes useful.

Instead of hard-coding tenant names into the model, the application can pass the resolved tenant into validation:

OrderCreate.model_validate(
    payload,
    context={"tenant": tenant},
)

The validator can access that information through ValidationInfo.

The important part for me isn't the specific API. It's the separation of responsibilities:

FastAPI dependency
        ↓
Resolve tenant
        ↓
TenantConfig
        ↓
Pydantic validation
        ↓
Validated request
        ↓
Endpoint

The endpoint doesn't need to know how any of that happened.

But there's a trade-off

I don't think this pattern is automatically the right answer for every FastAPI application.

To make tenant information available before Pydantic validation, the example intentionally receives the request body as raw data and calls model_validate() itself.

That gives us the context we need, but we give up some of FastAPI's automatic request-body handling.

One noticeable consequence is OpenAPI documentation.

With a normal Pydantic request model, FastAPI can automatically generate a detailed request schema. With the raw-body approach, the generated request schema is less descriptive.

That's a real trade-off.

If automatically generated API documentation were the most important requirement, I'd investigate a different integration.

What I actually care about

The goal isn't to eliminate every if statement from an application.

It's to keep responsibilities somewhere they make sense.

I'd rather have an endpoint that looks like:

def create_order(
    ctx: TenantOrderContext = Depends(get_validated_order),
) -> OrderResponse:
    ...

than one that starts doing all of this:

def create_order(request, tenant):
    # resolve tenant
    # check tenant status
    # determine limits
    # validate request
    # handle validation failures
    # process order

The second approach isn't inherently broken.

It's just carrying more responsibility than the endpoint needs to.

The broader lesson

The interesting part of this exercise wasn't really multi-tenancy.

It was thinking about where validation gets its information from.

Some rules depend only on the object being validated.

Others depend on the context surrounding that object.

When validation depends on external context, putting everything inside the route handler is an easy solution. But it isn't necessarily the cleanest boundary.

FastAPI dependencies and Pydantic v2 validation context give you another option.

I built a small working example of this approach with FastAPI, Pydantic v2, and pytest, including tests for tenant-specific limits, invalid requests, inactive tenants, unknown tenants, and malformed payloads.

The complete implementation is available on GitHub.

MaybeSomeone-arc18 / fastapi-multitenant-validation

Type-Safe Multi-Tenant Request Validation with FastAPI & Pydantic v2

A minimal, production-inspired educational Python example demonstrating type-safe multi-tenant request validation using FastAPI dependencies and Pydantic v2 validation context (info.context).

Purpose

Multi-tenant applications often need to validate incoming API payloads against tenant-specific rules (such as max order quantities, allowed product categories, or rate limits).

A common anti-pattern is placing if tenant == "acme": ... conditional branching inside route handlers. This pollutes endpoint logic and duplicates validation code.

This project demonstrates how to move tenant resolution and tenant-specific schema validation into FastAPI dependencies before the endpoint receives the request. The endpoint stays clean and receives a strongly-typed TenantOrderContext.

Request Flow & Architecture

Client
  │ (POST /orders, Header: X-Tenant-ID, JSON Body)
  ▼
1. get_tenant Dependency
  │ Reads X-Tenant-ID header & resolves TenantConfig
  │ Rejects missing header (422), unknown tenant (404), or inactive tenant (403)
  ▼
2. get_validated_order Dependency
  │ Receives

…

View on GitHub

Type-Safe Multi-Tenant Request Validation in FastAPI with Pydantic v2

Sanskar Kharya — Mon, 10 Aug 2026 09:42:31 +0000

Multi-tenant APIs often need to apply different validation rules depending on which tenant is making a request.

For example, imagine an order API where:

acme can create orders with up to 100 items.
globex can create orders with up to 10 items.

A straightforward implementation might put tenant-specific checks directly inside the endpoint:

if tenant == "acme":
    ...
elif tenant == "globex":
    ...

That works initially, but as the number of tenants and rules grows, the endpoint starts becoming responsible for both handling the request and understanding tenant-specific business rules.

This tutorial demonstrates a different approach using FastAPI dependencies and Pydantic v2 validation context.

The goal is to resolve the tenant before validating the request and pass that tenant configuration into Pydantic, while keeping the route handler free of tenant-specific validation logic.

What we're building

The request will look like this:

POST /orders
X-Tenant-ID: globex
Content-Type: application/json

{
  "product_id": "prod_123",
  "quantity": 5
}

The application will:

Resolve the tenant from X-Tenant-ID.
Reject unknown or inactive tenants.
Validate the request using Pydantic.
Apply the tenant's validation rules using Pydantic's validation context.
Pass a fully validated object to the endpoint.

The complete implementation is available in the fastapi-multitenant-validation GitHub repository.

Project structure

fastapi-multitenant-validation/
├── app/
│   ├── config.py
│   ├── dependencies.py
│   ├── main.py
│   └── models.py
├── tests/
│   ├── conftest.py
│   └── test_orders.py
├── pyproject.toml
└── README.md

The example intentionally keeps tenant configuration in memory. There is no database or external service because the goal is to demonstrate the validation architecture rather than build a complete multi-tenant platform.

Defining tenant configuration

First, we need a representation of a tenant.

from pydantic import BaseModel, Field


class TenantConfig(BaseModel):
    tenant_id: str = Field(..., description="Unique tenant identifier")
    name: str = Field(..., description="Human-readable tenant name")
    max_order_quantity: int = Field(
        ...,
        description="Maximum quantity allowed in a single order",
    )
    active: bool = Field(
        ...,
        description="Whether the tenant is currently active",
    )

For the example, we use a deterministic in-memory registry:

TENANTS: dict[str, TenantConfig] = {
    "acme": TenantConfig(
        tenant_id="acme",
        name="Acme Corporation",
        max_order_quantity=100,
        active=True,
    ),
    "globex": TenantConfig(
        tenant_id="globex",
        name="Globex Inc",
        max_order_quantity=10,
        active=True,
    ),
    "inactive_co": TenantConfig(
        tenant_id="inactive_co",
        name="Inactive Co",
        max_order_quantity=50,
        active=False,
    ),
}

The important part is that the tenant configuration contains information that can affect validation.

Defining the request model

The request-level constraints belong in our Pydantic model:

from pydantic import BaseModel, Field, ValidationInfo, model_validator


class OrderCreate(BaseModel):
    product_id: str = Field(..., min_length=1)
    quantity: int = Field(..., gt=0)

    @model_validator(mode="after")
    def validate_tenant_limits(
        self,
        info: ValidationInfo,
    ) -> "OrderCreate":
        if info.context and "tenant" in info.context:
            tenant: TenantConfig = info.context["tenant"]

            if self.quantity > tenant.max_order_quantity:
                raise ValueError(
                    f"Quantity {self.quantity} exceeds maximum allowed "
                    f"limit ({tenant.max_order_quantity}) for tenant "
                    f"'{tenant.tenant_id}'."
                )

        return self

These rules represent two different kinds of validation.

General request validation can be performed from the body alone:

product_id → must not be empty
quantity   → must be greater than zero

Tenant-specific validation needs information outside the body:

quantity → must not exceed tenant.max_order_quantity

Pydantic v2's validation context gives us a way to provide that external information during validation.

Resolving the tenant with a FastAPI dependency

Next, we resolve the tenant from the request header.

The repository uses FastAPI's required-header behavior directly:

from fastapi import Header, HTTPException


def get_tenant(
    x_tenant_id: str = Header(..., alias="X-Tenant-ID"),
) -> TenantConfig:
    tenant = TENANTS.get(x_tenant_id)

    if tenant is None:
        raise HTTPException(
            status_code=404,
            detail=f"Tenant '{x_tenant_id}' not found",
        )

    if not tenant.active:
        raise HTTPException(
            status_code=403,
            detail=f"Tenant '{x_tenant_id}' is inactive",
        )

    return tenant

Because the header is required, FastAPI itself handles a missing X-Tenant-ID header and returns a 422 Unprocessable Entity response.

For an unknown tenant, the dependency returns 404.

For an inactive tenant, it returns 403.

Most importantly, downstream dependencies now receive a strongly typed TenantConfig.

Connecting tenant resolution to Pydantic validation

This is where the two pieces come together.

We intentionally receive the request body as a raw dictionary:

payload: dict[str, Any] = Body(...)

That allows the tenant dependency to run before we explicitly invoke Pydantic validation.

The important part of the dependency is:

from pydantic import ValidationError
from fastapi import Body, Depends
from fastapi.exceptions import RequestValidationError


def get_validated_order(
    payload: dict[str, Any] = Body(...),
    tenant: TenantConfig = Depends(get_tenant),
) -> TenantOrderContext:
    try:
        order = OrderCreate.model_validate(
            payload,
            context={"tenant": tenant},
        )
    except ValidationError as exc:
        raise RequestValidationError(
            exc.errors(include_url=False)
        ) from exc

    return TenantOrderContext(
        tenant=tenant,
        order=order,
    )

The key line is:

OrderCreate.model_validate(
    payload,
    context={"tenant": tenant},
)

The tenant configuration is now available inside Pydantic's ValidationInfo.context.

Why catch `ValidationError`?

Normally, FastAPI automatically handles validation errors generated while processing request parameters.

Here, however, we are deliberately calling Pydantic ourselves inside a dependency.

That means we need to catch Pydantic's ValidationError and convert it into FastAPI's RequestValidationError. Otherwise, a validation failure could escape the normal request-validation handling and become a 500 Internal Server Error.

By converting it to RequestValidationError, the API consistently returns 422 Unprocessable Entity for both ordinary Pydantic validation failures and tenant-specific validation failures.

Keeping the endpoint clean

The endpoint receives a strongly typed TenantOrderContext:

@app.post(
    "/orders",
    response_model=OrderResponse,
    status_code=status.HTTP_201_CREATED,
)
def create_order(
    ctx: TenantOrderContext = Depends(get_validated_order),
) -> OrderResponse:
    return OrderResponse(
        order_id="ord_123456",
        tenant_id=ctx.tenant.tenant_id,
        product_id=ctx.order.product_id,
        quantity=ctx.order.quantity,
        status="accepted",
    )

There is no:

if tenant == "acme":

and no:

if tenant == "globex":

The endpoint receives a context whose tenant is already known to be active and whose order has already passed both baseline and tenant-specific validation.

That separation is the main architectural point of the example.

Testing the behavior

The repository includes eight tests covering successful requests and failure cases.

For example, Globex allows a maximum quantity of 10:

response = client.post(
    "/orders",
    headers={"X-Tenant-ID": "globex"},
    json={
        "product_id": "prod_123",
        "quantity": 5,
    },
)

assert response.status_code == 201

But a quantity of 15 should fail:

response = client.post(
    "/orders",
    headers={"X-Tenant-ID": "globex"},
    json={
        "product_id": "prod_123",
        "quantity": 15,
    },
)

assert response.status_code == 422

The test suite also covers:

invalid quantities
missing tenant headers
unknown tenants
inactive tenants
non-object JSON payloads

The complete suite passes successfully.

An important trade-off

There is a reason we don't simply declare:

order: OrderCreate

as the request body and let FastAPI handle everything automatically.

For this particular example, we need tenant information before performing tenant-aware Pydantic validation.

Receiving the raw body and explicitly calling:

OrderCreate.model_validate(
    payload,
    context={"tenant": tenant},
)

gives us that control.

The trade-off is that FastAPI's automatically generated OpenAPI documentation does not automatically expose the complete OrderCreate request schema in the same way that a normal Pydantic request-body parameter would.

This is intentional in the example. In a production system, you would need to decide whether this validation architecture or richer automatically generated API documentation is more important, or investigate an integration that provides both.

Conclusion

The main idea isn't that every FastAPI application should manually validate request bodies.

The useful pattern is separating responsibilities:

Request
   ↓
Tenant dependency
   ↓
TenantConfig
   ↓
Pydantic validation context
   ↓
Validated TenantOrderContext
   ↓
Clean endpoint

FastAPI dependencies handle request context, while Pydantic handles validation.

This becomes particularly useful when validation rules depend on information that isn't contained in the request body itself.

The complete, tested implementation is available in the fastapi-multitenant-validation GitHub repository.

MaybeSomeone-arc18 / fastapi-multitenant-validation

Type-Safe Multi-Tenant Request Validation with FastAPI & Pydantic v2

A minimal, production-inspired educational Python example demonstrating type-safe multi-tenant request validation using FastAPI dependencies and Pydantic v2 validation context (info.context).

Purpose

Multi-tenant applications often need to validate incoming API payloads against tenant-specific rules (such as max order quantities, allowed product categories, or rate limits).

A common anti-pattern is placing if tenant == "acme": ... conditional branching inside route handlers. This pollutes endpoint logic and duplicates validation code.

Request Flow & Architecture

Client
  │ (POST /orders, Header: X-Tenant-ID, JSON Body)
  ▼
1. get_tenant Dependency
  │ Reads X-Tenant-ID header & resolves TenantConfig
  │ Rejects missing header (422), unknown tenant (404), or inactive tenant (403)
  ▼
2. get_validated_order Dependency
  │ Receives

…

View on GitHub

DEV Community: Sanskar Kharya

The Guide That Sees for You: Building an On-Device Navigation Companion in the Browser

Why a phone in the cloud is too slow to keep you safe

What I actually built

From pixels to sound

1. Finding things: the detector

2. Turning a box into a sound you can locate

3. Telling a threat from a table: velocity vectoring

4. Reading the room: contextual modes

The honest gap: prototype vs. blueprint

Where it goes next

Why I Don't Put Tenant-Specific Validation in My FastAPI Endpoints

When the endpoint starts knowing too much

FastAPI dependencies are a natural boundary

Some validation needs context

But there's a trade-off

What I actually care about

The broader lesson

MaybeSomeone-arc18 / fastapi-multitenant-validation

Type-Safe Multi-Tenant Request Validation with FastAPI & Pydantic v2

Purpose

Request Flow & Architecture

Type-Safe Multi-Tenant Request Validation in FastAPI with Pydantic v2

What we're building

Project structure

Defining tenant configuration

Defining the request model

Resolving the tenant with a FastAPI dependency

Connecting tenant resolution to Pydantic validation

Why catch ValidationError?

Keeping the endpoint clean

Testing the behavior

An important trade-off

Conclusion

MaybeSomeone-arc18 / fastapi-multitenant-validation

Type-Safe Multi-Tenant Request Validation with FastAPI & Pydantic v2

Purpose

Request Flow & Architecture

Why catch `ValidationError`?