DEV Community: fenixkit

Three ways to serve files from S3 — and when to use each

fenixkit — Thu, 28 May 2026 10:43:10 +0000

When you add file storage to an API, the first decision is: how do you get those files to the client?

There are three fundamentally different strategies, each with different security properties, infrastructure requirements, and tradeoffs. Most tutorials pick one without explaining why. This article covers all three so you can choose the right one per use case — because in most real applications you'll need more than one.

The three strategies

1. Public URL

The file is stored in a public bucket. Anyone with the URL can access it — no authentication, no expiry, no server involvement after upload.

Client → CDN/S3 URL → File

When it makes sense:

Static assets that are genuinely public: logos, marketing images, open documentation
Files you'd serve from a CDN anyway
High-traffic read scenarios where you don't want your API in the critical path

Limitations:

Once the URL is out, it's out. You cannot revoke access to a specific file without deleting or moving it. If a user shares the URL, anyone can use it indefinitely. This rules it out for anything tied to permissions or subscriptions.

Also worth noting: if you later need to privatise a file, you have to migrate it to a different bucket with a different access policy. Plan your bucket layout with this in mind.

2. Presigned URL

The file lives in a private bucket. Your API generates a time-limited signed URL and returns it to the client. The client uses that URL directly to fetch the file — your server is not in the data path.

Client → API (auth check) → generates signed URL → Client → S3 URL (expires in N seconds)

When it makes sense:

Files that belong to a specific user or entity
Content that should expire (paid downloads, temporary access, trial periods)
Large files where you don't want to stream through your server

The key parameter: expiry. Too short and clients get errors mid-session. Too long and you've effectively made it public.

The cache problem — easy to miss:

If you cache the API response that contains the presigned URL, and that cache entry outlives the URL's expiry, clients will receive a cached response with a URL that no longer works. The fix is to align your cache TTL with the presigned URL expiry. If the URL expires in 1 hour, the cache entry must expire in at most 1 hour.

One infrastructure note: if your API runs in Docker and connects to a self-hosted S3 backend via an internal network address, presigned URLs need special handling. The signature is computed against the URL host, so if you sign with an internal Docker hostname, the resulting URL will contain that internal hostname — which the client cannot reach. You need to sign with the public-facing hostname.

3. Proxy

The file stays in a private bucket. The client never gets a storage URL at all — it requests the file through your API, your server fetches it from S3, and streams it back to the client.

Client → API (auth check) → S3 (internal) → streams back to Client

When it makes sense:

Files that require strict access control checked on every request
Scenarios where you need to log every access
Files where the storage URL must never leak (contracts, medical records, invoices)
Cases where you need to transform the file on the way out (watermarks, transcoding)

The cost:

Every download passes through your server. For large files or high traffic this adds bandwidth and compute costs, and your server becomes the bottleneck.

Use this mode when security requirements genuinely demand it, not as a default.

Comparison at a glance

	Public URL	Presigned URL	Proxy
Storage bucket	Public	Private	Private
Auth check on download	No	At generation time	On every request
URL expires	No	Yes (configurable)	N/A
Server in download path	No	No	Yes
Can revoke access	Only by deleting	Naturally (expiry)	Yes
CDN-friendly	Yes	Limited	No
Best for	Static assets	User content	Sensitive files

Per-bucket configuration

In most real applications you'll have multiple buckets, each warranting a different strategy. A typical setup might look like this:

public-assets — logos, marketing images → Public
user-uploads — profile pictures, product photos → PresignedUrl (7-day expiry)
private-documents — contracts, invoices → Proxy

The cleanest approach is to make the access mode a property of the bucket configuration rather than hardcoding it in business logic. Your file-handling code then reads the bucket config to decide how to resolve a download URL — no conditionals scattered through your services.

This also means changing a bucket's access mode is a config change, not a code change.

Choosing

A rough decision tree:

Is the file genuinely public and static? → Public URL
Does the file belong to a user but doesn't need per-request access checks? → Presigned URL
Does the file require access checked on every request, or must the URL never leak? → Proxy

When in doubt, start with Presigned URL — it gives you access control at generation time, natural expiry, and keeps your server out of the download path. Upgrade to Proxy only when you genuinely need per-request enforcement.

A note on provider choice

Everything above applies equally to AWS S3, Garage or any S3-compatible backend. The access mode strategy is independent of the provider — only the endpoint and credentials change. If you're self-hosting, Garage is worth a look: it's lightweight, runs in Docker, and is S3-compatible.

I implemented all three modes in FenixKit — a .NET Minimal API kit that comes with MongoDB, Keycloak, Redis, and S3 file storage pre-wired. Each bucket is configured independently with its own access mode. If you're building a .NET API and don't want to wire all this up from scratch, it might save you a few days. Public docs on GitHub

Why your .NET 8 API needs a cache layer — and how to build it right with Redis/Valkey and tag invalidation

fenixkit — Sun, 17 May 2026 18:18:44 +0000

Caching is one of those things that sounds optional until your database starts getting hammered at scale, your response times creep up, and you realise you've been querying the same data hundreds of times per minute. This article covers why a cache layer matters, how to implement cache-aside properly with tag-based invalidation in .NET 8, how to handle Redis outages gracefully, and why Valkey is worth knowing about.

Why bother with cache at all?

The short answer: your database doesn't need to answer the same question twice.

A typical read-heavy API hits the database for the same product list, the same user profile, the same category results — on every request. Each one is a network round trip, a query execution, and serialisation overhead. At low traffic it's fine. At scale it isn't.

A cache layer puts the answer in Redis the first time, and returns it directly on every subsequent request — milliseconds, no database involved.

The reasons people avoid it:

"It adds complexity" — only if you build it badly
"Cache invalidation is hard" — it is, but it doesn't have to be unpredictable
"Redis going down takes my API down" — only if you don't handle it properly

All three are solvable.

The cache-aside pattern

Cache-aside is the simplest correct approach:

On read — check Redis first. Hit → return. Miss → query the database, populate Redis, return.
On write — invalidate the relevant cache entries, then write to the database.

GET /api/products/abc123

  1. Check Redis  ──▶  HIT  ──▶  return cached JSON ✓
               └──▶  MISS ──▶  query database
                              └──▶  populate Redis ──▶  return ✓

PUT /api/products/abc123

  → invalidate cache entries for this product
  → write to database

Simple in theory. The problem is step 2 — which cache entries do you invalidate?

The invalidation problem

If you cache by key only (product:abc123), that's easy — delete that key on update. But most APIs cache more than that:

Paged lists — product:paged:p1:s20
Cursor pages — product:cursor:start:20:fwd
Filtered results — product:category:Gaming

When you update a product, all of those might be stale. You can't just delete one key.

The naive solution is to expire everything with a short TTL. It works, but it means serving stale data for up to N minutes after every write, and it doesn't scale — at high write rates your cache is constantly cold.

Tag-based invalidation

A better approach: every cached entry is registered under one or more tags. When you write, you invalidate by tag — wiping all entries associated with that tag at once.

In Redis, a tag is a Set that holds the keys registered under it:

product:abc123              STRING   cached product JSON          TTL 5 min
product:paged:p1:s20        STRING   cached page JSON             TTL 5 min
product:category:Gaming     STRING   cached category list         TTL 5 min

tag:product                 SET      { paged keys, cursor keys }    no TTL
tag:product:abc123          SET      { "product:abc123" }            no TTL
tag:product:category:Gaming SET      { "product:category:..." }      no TTL

Tag sets have no TTL — they are deleted when InvalidateByTagAsync runs, leaving no orphaned entries.

On every write, the repository wipes all matching tags.

The update case is worth calling out: when a product moves from Electronics to Gaming, you need to invalidate both the old and new category cache. The solution is to union the tags from the original and the updated entity before invalidating — both category caches get wiped, no extra logic needed in your handler.

Three levels of control

Not everything needs automatic invalidation. A well-designed cache layer gives you three levels:

Level	Mechanism	Use for
Automatic	Base repository calls `GetInvalidationTags` on every write	Standard CRUD — always on
Tag-based	`_cache.InvalidateByTagAsync("product:category:Gaming")`	Custom domain queries
Manual	`_cache.InvalidateAsync("product:abc123")`	Surgical single-key removal

You pick the right level per operation. Most of the time the automatic level handles everything.

Handling Redis outages — FailOpen vs FailClosed

This is where most cache implementations go wrong. If Redis throws an exception and you let it propagate, your API returns 500s whenever the cache is unavailable — even though your data is perfectly fine in the database.

FailOpen (recommended default): treat any Redis error as a cache miss. The request falls through to the database, succeeds, and returns normally. Redis being down is a performance degradation, not an outage.

FailClosed: return an error when Redis is unavailable. Use this only when cache correctness is a hard requirement.

For most APIs, FailOpen is the right default. Redis is a performance layer, not a source of truth.

Making cache optional

There are scenarios where you want to run without Redis entirely — local development or environments where you haven't provisioned a cache server yet.

The clean solution is a no-op implementation of your cache interface that can be swapped in via config:

// appsettings.json / .env
Cache__Enabled=false

When disabled: the cache interface resolves to a no-op, IConnectionMultiplexer is never registered, and the Redis health check is omitted automatically. No code changes required anywhere else.

Valkey — the Redis fork worth knowing about

In 2024, Redis changed its licence from BSD, no longer open-source. In response, the Linux Foundation forked Redis at version 7.2 and created Valkey — an open-source, community-maintained drop-in replacement.

Valkey is wire-protocol compatible with Redis. StackExchange.Redis connects to it transparently — no client changes, no code changes needed.

# docker-compose.valkey.yml
valkey:
  image: valkey/valkey:7.2-alpine
  command: valkey-server --requirepass ${CACHE_PASSWORD}
  ports:
    - "6379:6379"

valkey:6379,password=yourpassword,protocol=2

If you're happy with Redis 8, nothing changes. If you prefer a fully open-source stack, Valkey 7.2 is a transparent swap.

Putting it together

The full pattern in a .NET 8 Minimal API:

Read — check Redis, miss falls through to the database, result populates Redis on return
Write — union tags from old + new entity, invalidate, write to database
FailOpen by default — Redis errors never surface as 500s
Optional — disable via config, no-op swaps in automatically

If you'd rather not wire all of this from scratch, I've packaged the full implementation into FenixKit — .NET 8 Minimal API starter kits with the cache layer, tag invalidation, FailOpen, Valkey support, and health checks all included and pre-configured.

Why I stopped rolling my own auth and switched to Keycloak

fenixkit — Wed, 13 May 2026 12:49:06 +0000

Every developer has built it at least once. A UsersController, a POST /auth/login endpoint, a PasswordHasher, a JwtService that generates tokens. It feels like the natural thing to do — auth is just another feature, right?

It isn't. And I learned that the hard way.

What "rolling your own JWT auth" actually means

On the surface it looks simple:

var token = new JwtSecurityToken(
    issuer: "myapp",
    claims: claims,
    expires: DateTime.UtcNow.AddHours(1),
    signingCredentials: credentials);

But that's just the token. The moment you decide to own your auth stack, you're signing up for all of this:

Password storage — hashing, salting, choosing the right algorithm (bcrypt? Argon2? PBKDF2?), migrating if you get it wrong
Refresh token rotation — storing refresh tokens, invalidating old ones, handling concurrent requests that race to refresh
Token revocation — JWTs are stateless, so revoking one before expiry means a blacklist, which means a database lookup on every request
Brute force protection — rate limiting login attempts, lockout logic, alerting
Forgot password flow — secure token generation, expiry, one-time use enforcement
Email verification — same problem
MFA — TOTP, backup codes, recovery flows
Session management — single sign-out, concurrent session limits
Security patches — when a vulnerability is found in your approach, you fix it

Together they are a significant, ongoing maintenance burden — and they have nothing to do with your actual product.

The moment I realised I was building an identity provider

I was three days into a project without doing a single line of domain code.

That's when it clicked. I wasn't building a feature. I was building an identity provider — badly, from scratch, under time pressure. Companies spend years hardening this stuff.

The question isn't "can I build this?" — of course you can. The question is "should I?"

What Keycloak actually is

Keycloak is an open-source identity and access management solution. It handles everything in the list above — and more — out of the box. You run it as a container, configure a realm, and your application stops caring about any of it.

Your API's only job becomes: validate the token.

// This is the entire auth setup in your API
builder.Services
    .AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options =>
    {
        options.Authority = "http://localhost:8080/realms/myrealm";
        options.Audience  = "my-api-client";
    });

That's it. ASP.NET Core fetches the OIDC discovery document from Keycloak (/.well-known/openid-configuration), and validates every incoming token. No database lookup per request. No code to maintain.

But what about "JWT normal"?

When people say "just use JWT" they usually mean: generate tokens yourself, validate them yourself, store user data yourself. This is fine for a toy project or a quick internal tool.

The problem is that JWT is a token format, not an auth system. It tells you how to structure and sign a token. It tells you nothing about:

How to manage users
How to handle token revocation
How to implement refresh flows securely
How to add MFA later without rewriting everything

Keycloak uses JWT — it just handles all the surrounding complexity so you don't have to.

The honest trade-offs

Keycloak is not the right answer for every situation.

	Roll your own	Keycloak
Setup time	Fast initially	Slower initially
Maintenance	You own everything	Keycloak team maintains it
Flexibility	Total control	Configurable but opinionated
Resource usage	Minimal	Needs a container
MFA, SSO, social login	Build it yourself	Already there
Security patches	Your problem	Keycloak's problem

For a side project with no users yet — rolling your own might be fine. For anything with real users, a team, compliance requirements, or plans to grow — Keycloak wins on every axis that matters.

The .NET integration gotcha nobody warns you about

If you switch from rolling your own JWT to Keycloak in ASP.NET Core, there's one thing that will silently break: role claims.

ASP.NET Core remaps JWT claim names by default. sub becomes ClaimTypes.NameIdentifier, email becomes a long URN string — and roles gets ignored entirely because Keycloak puts realm roles in a non-standard location.

The fix is two lines, but you have to know to add them:

// Before builder.Services.AddAuthentication()
JwtSecurityTokenHandler.DefaultInboundClaimTypeMap.Clear();
JsonWebTokenHandler.DefaultInboundClaimTypeMap.Clear();

Without this, User.IsInRole("admin") always returns false and you spend an afternoon debugging a problem that isn't in your code.

You also need to tell the JWT Bearer middleware where Keycloak puts roles (in my case):

options.TokenValidationParameters = new()
{
    RoleClaimType = "roles",
    NameClaimType = "preferred_username",
    // ...
};

With these in place, [Authorize(Roles = "admin")] and User.IsInRole("admin") work exactly as expected.

The result

Once it's wired up, protecting any endpoint is a single line:

group.MapGet("/",        GetAll).RequireAuthorization("Authenticated");
group.MapDelete("/{id}", Delete).RequireAuthorization("AdminOnly");

Forgot password, MFA, social login, token revocation, brute force protection — all handled. You never wrote a PasswordHasher. You never debugged a refresh token rotation race condition. You just built your product.

Where to go from here

If you want to try Keycloak with .NET 8, the official docs are a reasonable starting point.

I spent time getting all of this right and packaged it into a starter kit — FenixKit MongoDB + Keycloak Edition — a .NET 8 Minimal API template with Keycloak pre-configured, a pre-built realm that imports at startup, and two test users ready to go. If you want to skip the setup and get straight to building, it's at fenixkit.dev.

If you want to find out more on how I built it, go to FenixKit GitHub.

Why I stopped using exceptions for control flow in my .NET 8 APIs

fenixkit — Sat, 09 May 2026 17:36:59 +0000

Why I stopped using exceptions for control flow in my .NET 8 APIs

For a long time, my .NET APIs looked like this:

public async Task<Product> GetByIdAsync(string id)
{
    var product = await _collection.Find(x => x.Id == id).FirstOrDefaultAsync();

    if (product is null)
        throw new KeyNotFoundException($"Product with id '{id}' was not found.");

    return product;
}

And in the endpoint:

app.MapGet("/api/products/{id}", async (string id, IProductRepository repo) =>
{
    try
    {
        var product = await repo.GetByIdAsync(id);
        return Results.Ok(product);
    }
    catch (KeyNotFoundException ex)
    {
        return Results.NotFound(ex.Message);
    }
    catch (Exception ex)
    {
        return Results.Problem(ex.Message);
    }
});

It worked. But every endpoint had a try/catch. Every service method threw a different exception type. The caller had to know which exceptions to catch. Business logic and error routing were tangled together.

Then I came across the ErrorOr library by Amantinband, and it changed how I think about error handling entirely.

The problem with exceptions for control flow

Exceptions in .NET are designed for exceptional situations — things that should not happen and cannot be recovered from gracefully. Using them to signal "product not found" or "name already exists" is semantically wrong and has real costs:

Performance — throwing and catching exceptions is significantly more expensive than returning a value. The CLR unwinds the call stack, captures a stack trace, and allocates an exception object every time.
Hidden contracts — nothing in the method signature tells the caller what can go wrong. You find out at runtime, not compile time.
Scattered try/catch — every caller has to know which exceptions to catch, and if they forget one, it bubbles up as an unhandled 500.
Hard to test — asserting on thrown exceptions is more awkward than asserting on return values.

The alternative is to make errors part of the return type. This is not a new idea — it is how languages like Rust (Result<T, E>) and Go (multiple return values) handle it. In .NET, the ErrorOr library brings the same pattern with a clean API.

What ErrorOr looks like

ErrorOr<T> is a discriminated union that holds either a success value or one or more errors. Every operation either succeeds or fails — and the caller is forced to handle both cases.

Here is what my domain error definitions look like in FenixKit:

public static partial class ProductErrors
{
    public static Error NotFound(string id) =>
        Error.NotFound(
            code: "Product.NotFound",
            description: $"Product with id '{id}' was not found.");

    public static Error NameConflict(string name) =>
        Error.Conflict(
            code: "Product.NameConflict",
            description: $"A product with name '{name}' already exists.");

    public static readonly Error InvalidPrice =
        Error.Validation(
            code: "Product.InvalidPrice",
            description: "Price must be greater than zero.");

    public static readonly Error InvalidCategory =
        Error.Validation(
            code: "Product.InvalidCategory",
            description: "Category cannot be empty.");
}

Each error has a type (NotFound, Conflict, Validation), a machine-readable code, and a human-readable description. No exception classes, no inheritance hierarchy, no throwing.

How it flows through the stack

The repository

Validation runs before the database is touched. If anything fails, we return an error and the operation stops:

public override async Task<ErrorOr<Success>> OnValidateCreateAsync(
    ProductCreateRequest request, CancellationToken ct = default)
{
    List<Error> errors = [];

    if (request.Price <= 0)
        errors.Add(ProductErrors.InvalidPrice);

    if (string.IsNullOrWhiteSpace(request.Category))
        errors.Add(ProductErrors.InvalidCategory);

    // Return all validation errors at once — no need to make the
    // client fix one problem, resubmit, and discover the next one.
    if (errors.Count > 0)
        return errors;

    // Only hit the database for uniqueness check if format validation passed.
    // This avoids a round-trip when the request is obviously invalid.
    var exists = await ExistsByNameAsync(request.Name, ct);
    if (exists.IsError)
        return exists.Errors;

    return exists.Value
        ? ProductErrors.NameConflict(request.Name)
        : Result.Success;
}

The hook chain

Every hook in BaseRepository returns ErrorOr<T>, so a failure at any stage aborts the operation cleanly before the database is touched:

public virtual async Task<ErrorOr<TDetailResponse>> CreateAsync(
    TCreateRequest request, CancellationToken ct = default)
{
    // 1. Validate — aborts if invalid
    var validation = await OnValidateCreateAsync(request, ct);
    if (validation.IsError) return validation.Errors;

    // 2. Map and enrich — aborts if enrichment fails
    var entity = request.ToDBEntity();
    var enriched = await OnMapCreateAsync(request, entity, ct);
    if (enriched.IsError) return enriched.Errors;

    // 3. Persist — aborts if the DB call fails
    var created = await _Repository.CreateAsync(enriched.Value, ct);
    if (created.IsError) return created.Errors;

    // 4. Project to response — aborts if projection fails
    var detail = await OnMapToDetailAsync(created.Value, ct);
    if (detail.IsError) return detail.Errors;

    return detail.Value;
}

No try/catch anywhere. Each step either produces a value or returns errors — and errors short-circuit the chain.

The endpoint

At the endpoint layer, Match forces you to handle both cases. There is no way to accidentally ignore the error path:

private static async Task<IResult> Create(
    ProductCreateRequest dto,
    [FromServices] IProductRepository repo,
    CancellationToken ct)
{
    var result = await repo.CreateAsync(dto, ct);

    return result.Match(
        created => Results.Created($"/api/products/{created.Id}", created),
        errors  => errors.ToResponse());
}

errors.ToResponse() maps each ErrorOr error type to the correct HTTP status:

public static IResult ToResponse(this List<Error> errors)
{
    // All validation errors → 422 with all field errors grouped
    if (errors.All(e => e.Type == ErrorType.Validation))
    {
        return Results.ValidationProblem(
            errors.ToDictionary(
                e => e.Code,
                e => new[] { e.Description }), statusCode: 422);
    }

    var first = errors.First();

    return first.Type switch
    {
        ErrorType.NotFound     => Results.NotFound(ToProblem(first, 404)),
        ErrorType.Conflict     => Results.Conflict(ToProblem(first, 409)),
        ErrorType.Unauthorized => Results.Unauthorized(),
        ErrorType.Forbidden    => Results.Forbid(),
        _                      => Results.Problem(ToProblem(first, 500).Detail)
    };
}

The mapping is defined once, used everywhere. No endpoint needs to know about HTTP status codes — it just calls ToResponse().

What the response looks like

A validation failure returns a proper RFC 7807 response:

{
  "type": "https://tools.ietf.org/html/rfc7807",
  "status": 422,
  "errors": {
    "Product.InvalidPrice": ["Price must be greater than zero."],
    "Product.InvalidCategory": ["Category cannot be empty."]
  }
}

All errors are returned at once — not one at a time. The client fixes everything in one round-trip.

A not-found error returns:

{
  "status": 404,
  "title": "Product.NotFound",
  "detail": "Product with id '6638f1a2b3c4d5e6f7a8b9c0' was not found."
}

Machine-readable code, human-readable description, correct HTTP status. No exceptions needed.

The global exception handler as a safety net

ErrorOr handles expected errors — things your domain knows can go wrong. But bugs happen. For everything truly unexpected, a global exception handler catches it and converts it to a ProblemDetails response before it reaches the client:

public async ValueTask<bool> TryHandleAsync(
    HttpContext context, Exception exception, CancellationToken ct)
{
    var (statusCode, title) = exception switch
    {
        KeyNotFoundException      => (404, "Resource not found"),
        UnauthorizedAccessException => (401, "Unauthorized"),
        ArgumentException         => (400, "Bad request"),
        InvalidOperationException => (409, "Conflict"),
        _                         => (500, "Internal server error")
    };

    var problem = new ProblemDetails
    {
        Status   = statusCode,
        Title    = title,
        Detail   = exception.Message,
        Instance = context.Request.Path
    };

    // TraceId for correlation — useful when the client reports a bug
    problem.Extensions["traceId"] = context.TraceIdentifier;

    context.Response.StatusCode      = statusCode;
    context.Response.ContentType     = "application/problem+json";

    await context.Response.WriteAsJsonAsync(problem, ct);
    return true;
}

No stack traces leak to the client. No 500 with a raw exception message. The traceId lets you correlate a client report with your logs.

Summary

Switching to ErrorOr changed three things in my APIs:

Errors are explicit. The return type tells you what can go wrong. You do not discover error cases at runtime.

No scattered try/catch. The only try/catch in the whole codebase is the global exception handler — which exists for genuine bugs, not business logic.

HTTP mapping is centralised. ToResponse() is defined once. Every endpoint uses it. Adding a new error type means updating one switch, not every endpoint that could encounter it.

The pattern does require a small mindset shift — you stop thinking of errors as exceptional events and start treating them as valid return values. Once that clicks, it is hard to go back.

All code in this article is taken directly from FenixKit — a ready .NET 8 Minimal API starter kit with MongoDB that I built and packaged to stop rewriting the same foundation on every project.

References

ErrorOr library by Amantinband — github.com/amantinband/error-or
RFC 7807 — Problem Details for HTTP APIs — tools.ietf.org/html/rfc7807
Microsoft Docs — Exception handling in .NET — learn.microsoft.com/dotnet/standard/exceptions
Microsoft Docs — IExceptionHandler in ASP.NET Core — learn.microsoft.com/aspnet/core/fundamentals/error-handling
Microsoft Docs — Minimal APIs in ASP.NET Core — learn.microsoft.com/aspnet/core/fundamentals/minimal-apis