DEV Community: OtezVikentiy

How to upload a file over JSON-RPC, when JSON has no type for a file

OtezVikentiy — Wed, 19 Aug 2026 06:40:00 +0000

JSON has no representation for a file. Strings, numbers, arrays, objects - that is the whole list. So every JSON-RPC API eventually runs into the same question: how do you accept a file upload - a photo, a scan, a PDF - when the protocol itself cannot carry binary data?

The usual answer is: you don't. The file goes to a separate, ordinary controller that reads $request->files, and the JSON-RPC layer handles everything else next to it. And now you have exactly the ad hoc endpoint sprawl that JSON-RPC was supposed to remove.

In otezvikentiy/json-rpc-api 5.2 there is a different answer. And more interesting than the feature is how it came about: I did not write it - an external contributor did. But first things first.

Full disclosure: I am the author of the bundle, and I have maintained it alone for almost three years. That is exactly why a release whose headline feature was written by someone else feels like a different kind of event to me.

The problem

Straight from issue #8: two services exchange scanned images plus structured metadata (tenant, station, session) on the same call - something like captures.create(tenantId, stationId, image). Today image cannot be expressed as a parameter of a JSON-RPC method, so that call has to live outside the bundle as a separate multipart controller.

What you want is for the method to simply declare a parameter of type UploadedFile and get the file, like any other parameter.

The solution: multipart as a transport adapter

The key idea is to leave the core untouched. A multipart/form-data request is normalized into the very same JSON-RPC envelope an ordinary request produces, only with UploadedFile objects already sitting inside params. Everything below the transport - hydration, batching, validation - stays completely unaware of multipart, exactly the way it is unaware that a GET request's payload came from a query string.

The wire format: one text part named jsonrpc carries the full JSON-RPC envelope as a string (all scalar parameters live inside it), and every other part is a file, its part name being the parameter name.

curl -X POST http://localhost/api/v1 \
  -F 'jsonrpc={"jsonrpc":"2.0","method":"captures.create","params":{"tenantId":"t-1"},"id":1}' \
  -F 'image=@scan.png'

The method declares the parameter as an ordinary DTO property:

use Symfony\Component\HttpFoundation\File\UploadedFile;

final class Request
{
    private string $tenantId = '';
    private ?UploadedFile $image = null;

    public function getTenantId(): string { return $this->tenantId; }
    public function setTenantId(string $tenantId): void { $this->tenantId = $tenantId; }

    public function getImage(): ?UploadedFile { return $this->image; }
    public function setImage(?UploadedFile $image): void { $this->image = $image; }
}

And reads it in the handler as a real UploadedFile - with move(), getClientOriginalName(), the whole Symfony surface.

Three decisions worth explaining

Scalars do not become form fields. The temptation was to spread every parameter across separate form fields. But a form field is a string, and then "42" and 42 become indistinguishable again - the untyped-transport ambiguity the bundle deliberately tolerates only for GET, where a query string leaves no choice. POST has types, and losing them is not worth it. So scalars stay in the JSON envelope, and only files travel as form parts.

It is switched on twice. One switch is multipart.enabled for the application, the other is acceptsMultipart: true on the method's attribute. This is not belt-and-suspenders: the Content-Type is checked before any method is known, so the global switch alone cannot say anything about a particular method. And turning the transport on for the application while silently opening it to every method already written - none of which expected it - is the wrong default.

Validation is Symfony's own, not hand-rolled. A declared UploadedFile compiles to Assert\Type followed by Assert\File, through the same machinery that produces Assert\Type('int') for an int field. The size limit (multipart.max_file_bytes, in Symfony's own notation such as '10Mi') is enforced by Assert\File, and it brings the handling of all eight PHP upload error codes with it. A failed upload (upload_max_filesize exceeded, a partial transfer, no temp directory) comes back as -32602 naming the field, rather than as an unusable UploadedFile reaching the method.

Honest about security

multipart/form-data is a CORS "simple request", exactly like form-encoded. And the mandatory Content-Type: application/json introduced in 5.0 was precisely what closed that CSRF vector. So enabling multipart reopens it - but for the methods that declare acceptsMultipart: true, and only those.

The bundle does not pretend it solved this for you. The docs warn loudly: before switching it on, make sure at least one of these holds for the affected methods - authentication does not travel in cookies (a header token is not CORS-safelisted), or the session cookie is SameSite=Lax/Strict, or the method checks a CSRF token. The two-level opt-in limits the blast radius; the rest is the application's decision, not the bundle's.

The first-version limits are stated plainly too: batch stays JSON-only, files at the top level of params only, POST only.

How it came about - and why that matters more than the feature

I did not write this code. It started with issue #8: tacman described the problem, proposed two shapes (a minimal one and one modeled on the GraphQL multipart spec), mapped out honestly where it would touch hydration, and asked for direction before writing anything. We settled the shape in the comments. A few days later came PR #9: seven commits, a green CI across the whole matrix including the coverage and mutation-testing gates, plus a branch on the demo application so the feature could be run rather than only read.

In a couple of places his solution was better than the sketch in the issue - in particular compiling Assert\File from the config, which I had not planned. The review took one pass: run his branch locally (822 tests, Infection MSI above the gate, PHPStan and cs-fixer clean), read the whole diff, and leave a few non-blocking notes. Merging was not scary.

For a maintainer who carried the project alone for three years, a first serious external PR - a careful one, with tests and a demo - is worth more than any number of stars. It is the first sign that something living is forming around the project, and probably the best outcome one could hope for when opening the source. Thank you, tacman.

Install and try it

composer require otezvikentiy/json-rpc-api:^5.2

The full release: 5.2 on GitHub.
Feature docs (wire format, config, the error catalogue, and the patterns for cases outside its scope - base64 for small payloads, a two-step upload for large ones): docs/multipart.md.
The demo project where it all works together: symfony-jsonrpc-api-demo.

The bundle: github.com/OtezVikentiy/symfony-jsonrpc-api-bundle. Questions and ideas in Discussions, bugs in Issues. As this story shows, a good issue sometimes turns into a feature - feedback of any kind is welcome.

Build a JSON-RPC 2.0 API in Symfony in 15 minutes: from composer require to OpenAPI

OtezVikentiy — Tue, 11 Aug 2026 06:35:02 +0000

REST works great while your API describes resources. But as soon as the domain becomes verb-shaped - recalculateInvoice, mergeAccounts, assignTask - you end up bending verbs into nouns and arguing about which HTTP method cancels an order. JSON-RPC 2.0 cuts through all of that: every call is just method + params, one endpoint, a spec that fits on two pages, and batching out of the box.

In this article we will build a working JSON-RPC 2.0 API on Symfony: a task tracker with DTO validation, batch requests and generated OpenAPI documentation. There is surprisingly little code to write: methods are declared with attributes, validation is derived from property types, and Swagger is generated by a console command.

Everything below lives as a ready-to-run project on GitHub: symfony-jsonrpc-api-demo - clone it and poke it with curl while you read. We will use the otezvikentiy/json-rpc-api bundle (PHP 8.2-8.5, Symfony 6.4/7/8; this article uses PHP 8.4 and Symfony 7.4).

Full disclosure: I am the author of the bundle. It has been running in production for three years - internal fintech tooling, an HRM system - nothing glamorous load-wise, but the correctness, logging and audit requirements were real, and they shaped most of what you will see below.

Installation

composer create-project symfony/skeleton:"7.4.*" tasks-api
cd tasks-api
composer require otezvikentiy/json-rpc-api

If Flex has contrib recipes enabled, the bundle registers itself. If not, it is two lines by hand:

// config/bundles.php
return [
    // ...
    OV\JsonRPCAPIBundle\OVJsonRPCAPIBundle::class => ['all' => true],
];

Wire up the route and a minimal config:

# config/routes/ov_json_rpc_api.yaml
ov_json_rpc_api:
    resource: '@OVJsonRPCAPIBundle/config/routes/routes.yaml'

# config/packages/ov_json_rpc_api.yaml
ov_json_rpc_api:
    access_control_allow_origin_list:
        - 'http://localhost:8000'

The bundle registers a single route, /api/v{version} - every request goes through it. Note the CORS list format: these are full origins, scheme://host[:port], exactly as the browser sends them in the Origin header.

Check that it is alive:

php -S localhost:8000 -t public
curl -s -X POST http://localhost:8000/api/v1 \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"test","params":{},"id":1}'

{"jsonrpc":"2.0","error":{"code":-32601,"message":"Method not found."},"id":1}

A well-formed JSON-RPC "method not found" error means the transport works. Now let's add methods.

The first method: createTask

A method in this bundle is three classes: a Request (what comes in), a Response (what goes out) and the method itself carrying the #[JsonRPCAPI] attribute. No YAML manifests, no base controllers to extend.

The Request describes the parameters. The rule is simple: constructor parameters are required, properties with setters are optional:

// src/RPC/V1/CreateTask/CreateTaskRequest.php
declare(strict_types=1);

namespace App\RPC\V1\CreateTask;

final class CreateTaskRequest
{
    private ?string $assigneeEmail = null;

    public function __construct(private string $title)
    {
    }

    public function getTitle(): string
    {
        return $this->title;
    }

    public function setTitle(string $title): void
    {
        $this->title = $title;
    }

    public function getAssigneeEmail(): ?string
    {
        return $this->assigneeEmail;
    }

    public function setAssigneeEmail(?string $assigneeEmail): void
    {
        $this->assigneeEmail = $assigneeEmail;
    }
}

The Response is a plain class; whatever the class makes public - a public getter or a public property - ends up in the JSON. Promoted constructor properties keep it short:

// src/RPC/V1/CreateTask/CreateTaskResponse.php
declare(strict_types=1);

namespace App\RPC\V1\CreateTask;

final class CreateTaskResponse
{
    public function __construct(
        public readonly bool $success,
        public readonly int $id,
        public readonly string $title,
        public readonly string $status,
        public readonly ?string $assigneeEmail,
    ) {
    }
}

And the method itself - an ordinary autowired service:

// src/RPC/V1/CreateTaskMethod.php
declare(strict_types=1);

namespace App\RPC\V1;

use App\RPC\V1\CreateTask\CreateTaskRequest;
use App\RPC\V1\CreateTask\CreateTaskResponse;
use App\Task\TaskStorage;
use OV\JsonRPCAPIBundle\Core\Annotation\JsonRPCAPI;
use OV\JsonRPCAPIBundle\Core\ApiMethodInterface;

#[JsonRPCAPI(
    methodName: 'createTask',
    type: 'POST',
    summary: 'Create a task',
    tags: ['tasks'],
)]
final class CreateTaskMethod implements ApiMethodInterface
{
    public function __construct(private readonly TaskStorage $storage)
    {
    }

    public function call(CreateTaskRequest $request): CreateTaskResponse
    {
        $task = $this->storage->create($request->getTitle(), $request->getAssigneeEmail());

        return new CreateTaskResponse(true, $task->id, $task->title, $task->status->value, $task->assigneeEmail);
    }
}

TaskStorage here is a trivial JSON-file store (about 80 lines in the demo repo; in a real project a Doctrine repository takes its place - the method contract does not change). The API version is derived from the namespace: App\RPC\V1 -> /api/v1.

Call it:

curl -s -X POST http://localhost:8000/api/v1 \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"createTask","params":{"title":"Try the demo","assigneeEmail":"alice@example.com"},"id":1}'

{"jsonrpc":"2.0","result":{"success":true,"id":1,"title":"Try the demo","status":"todo","assigneeEmail":"alice@example.com"},"id":1}

Validation you don't have to write

The best part: the bundle builds the validator set itself, from the PHP types of the Request class. private string $title in the constructor means "required, string". ?string $assigneeEmail with a setter means "optional, string or null". Since version 5.0 the type comparison is strict - no silent coercion:

curl -s -X POST http://localhost:8000/api/v1 \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"createTask","params":{"title":123},"id":4}'

{"jsonrpc":"2.0","error":{"code":-32602,"message":"Invalid params. Additional info: [title] - This value should be of type string."},"id":4}

"123" will not quietly become a number, 1 will not become true - the client gets an immediate -32602 Invalid params with a readable explanation. Your request contract is literally the PHP types of your DTO.

Domain errors: JRPCException

For errors like "task not found" the bundle ships JRPCException - a thrown exception becomes a proper JSON-RPC error object:

// src/RPC/V1/GetTaskMethod.php (excerpt)
public function call(GetTaskRequest $request): GetTaskResponse
{
    $task = $this->storage->find($request->getId());
    if ($task === null) {
        throw new JRPCException('Task not found.', JRPCException::SERVER_ERROR);
    }

    return new GetTaskResponse(true, $task->id, $task->title, $task->status->value, $task->assigneeEmail);
}

{"jsonrpc":"2.0","error":{"code":-32000,"message":"Task not found."},"id":1}

The spec-defined codes (-32700...-32603) and the server range [-32099, -32000] are validated by the exception itself - you cannot accidentally invent an invalid code. Everything else - any unexpected Throwable - reaches the client as a generic -32603 Internal error, while the full stack trace goes to the log only. No leaking file paths or class names to the outside world.

Batches: N calls, one HTTP request

This is a protocol feature rather than a bundle feature, but here it works out of the box - send an array of requests instead of a single object:

curl -s -X POST http://localhost:8000/api/v1 \
  -H "Content-Type: application/json" \
  -d '[{"jsonrpc":"2.0","method":"createTask","params":{"title":"One"},"id":1},
       {"jsonrpc":"2.0","method":"createTask","params":{"title":"Two"},"id":2},
       {"jsonrpc":"2.0","method":"listTasks","params":{},"id":3}]'

The response is an array of three results, matched by id. Where a REST client makes N round-trips (and the frontend shows N spinners), this is one request. The maximum batch size is capped by config (max_batch_size), so a million-call batch DoS does not get through.

OpenAPI from the same source of truth

The method attributes and DTO types are the single source of truth - and the OpenAPI 3.1 document is generated from them:

# config/packages/ov_json_rpc_api.yaml (add this)
ov_json_rpc_api:
    # ...
    swagger:
        api_v1:
            api_version: '1'
            base_path: 'http://localhost:8000'
            info:
                title: 'Tasks JSON-RPC API'
                description: 'Task tracker built on JSON-RPC 2.0'

# .env
OV_JSON_RPC_API_SWAGGER_PATH=public/openapi/

php bin/console ov:swagger:generate

The output is public/openapi/api_v1.yaml with schemas for every Request/Response - ready for Swagger UI, Postman or client SDK generation. The docs cannot drift away from the code, because they are made from it.

What else is in the demo

The demo repo takes the same application further, and you can see the rest working there:

the full CRUD - getTask, listTasks with a status filter, updateTask, deleteTask;
API versioning: a paginated listTasks v2 lives at /api/v2 without touching v1 - it is just a second namespace, App\RPC\V2;
sane default limits: body size, JSON depth, batch size - all configurable, all covered by tests;
request logging with masking of sensitive fields by regex patterns (29 built-in patterns: password, token, card_number, ...);
functional tests through a regular WebTestCase - 26 tests over the real endpoint.

Each of those deserves its own write-up - tell me in the comments which one to start with.

My idle ClickHouse was merging 11 million rows every 30 seconds

OtezVikentiy — Fri, 24 Jul 2026 21:12:14 +0000

I run a small self-hosted observability tool on the cheapest VPS I could find on purpose: 2 cores, 2 GB RAM, 20 GB SATA SSD. It ingests errors, traces and metrics from two low-traffic sites of mine. The stack is three containers — a Go app, PostgreSQL, and ClickHouse.

One evening docker stats showed ClickHouse sitting on 880 MB of its 1 GB limit and the box swapping, with basically zero events coming in. So I went looking for where the memory and disk had gone. The answer turned out to be a good lesson in how a database can spend almost all of its I/O talking to itself.

543 KB of my data, 579 MB of ClickHouse talking about ClickHouse

First thing I checked: how much data had my app actually stored versus how much ClickHouse had stored about itself.

My application database: 543 KB, 16k rows
The system database: 579 MB, 46.3M rows

Roughly a thousand to one. Disk was 12 GB used out of 20 — on a tool that had recorded half a megabyte of real telemetry.

The culprit was ClickHouse's own system logs, several of which have no TTL by default and therefore grow forever:

trace_log — 404 MB, 26M rows (the query profiler writes here; it's on by default, sampling once per second)
asynchronous_metric_log — 16.6M rows
text_log — 132 MB
plus query_log, latency_log

Only metric_log, processors_profile_log and part_log ship with a TTL. Everything else just accumulates.

Then I looked at the insert rate over 30 seconds:

trace_log — 227 rows/s
asynchronous_metric_log — 157 rows/s
text_log — 44 rows/s
my application — about 5 rows/s

98.8% of all inserts were ClickHouse narrating its own internals.

The part that's expensive beyond disk

Here's the number that made me stop. Over the same 30 seconds:

rows inserted: 16,222
rows merged: 11,007,643

That's a 1 : 678 ratio. For every row written, the engine rewrote 678 already-sitting rows.

The mechanics: MergeTree drops every insert into its own data part, then merges parts into bigger ones so reads stay fast. When the table is small this is cheap. But when a table holds 26M rows and the inserts are tiny and constant, each successive merge drags along more and more already-written data. In the limit, the engine spends most of its I/O shuffling old rows around, not storing new ones. That was my "60% disk busy, idle app" mystery.

The hypothesis that was wrong

I was sure I had it: the bloated trace_log drives the merges, the merges burn CPU, and the profiler samples the merge threads too — a nice closed loop. Test: measure CPU, TRUNCATE TABLE system.trace_log, measure again.

CPU didn't drop. ~69% before, ~81% after.

Honest caveat: my "after" window opened 60 seconds after deleting 400 MB, and part cleanup itself loads the machine, so some of that rise could be the TRUNCATE. But the point stood — one table didn't explain it.

So I disabled the heavy logs entirely and re-measured. Merges collapsed:

merged rows / 30s: 11,007,643 → 5,727
inserted rows / 30s: 16,222 → 35
disk: 2.7 GB freed out of 20

And CPU… barely moved: 69% → 61%.

The conclusion I had to accept: the system logs were a real disk and I/O problem, but they were not the CPU cause. Two separate symptoms I'd carelessly glued into one.

The trap I set for myself

While writing this up I'd claimed ClickHouse was "eating half the machine." It wasn't, and the mistake is a common one: docker stats reports CPU as a percentage of one core, not the whole machine. 60% in docker stats on a 2-core box is ~30% of the box. ps on the host confirmed it: 51.4% across two cores. If you debug container load, always cross-check against the host — it's easy to double your own problem on paper.

What actually moved the needle

Disabling the heavy logs, plus tuning for a small machine:

asynchronous_metrics_update_period_s: 1 → 60
metric_log: collect 1s → 30s, flush 7.5s → 60s, 3-day TTL
background pools: background_schedule_pool_size 128 → 8 (the default targets many-core servers), background_pool_size 4, the rest at 2
mark_cache_size: 5 GiB → 256 MiB. Yes — the default mark cache is five times the whole container's memory limit.

Result, averaged over 15 samples:

	Before	After
Container memory	842–920 MB	256 MB
Host memory used	1043 MB	779 MB
Load average	1.15	0.79
Disk used	12 GB	9.2 GB

The big win is memory: −66%. ClickHouse had been living at 85–90% of its cgroup limit and would OOM on any spike; now it sits at ~25% with real headroom. The mark cache is most of that.

The mistake that took prod down

Worth telling, because it's the one that stings. I applied the first tuning config straight to the live server without testing it locally. ClickHouse runs a sanity check at startup: number_of_free_entries_in_pool_to_execute_mutation (default 20) must not exceed background_pool_size × background_merges_mutations_concurrency_ratio. I'd set background_pool_size=4, the product was 8, the check failed → BAD_ARGUMENTS → the server refused to start. Monitoring was down for a few minutes.

The bug wasn't the number. It was the order of operations. The fix — which is also how I recovered — is to boot a throwaway container of the same version with the config locally, confirm it starts, then ship. It takes thirty seconds, which is less than a prod rollback. The rollback, thankfully, was clean: a .bak next to the file, restored in 8 seconds.

Don't tune your defaults down to the minimum

When I brought these settings into the project repo, someone reasonably asked: won't they hurt a 10-core / 10 GB server? Yes, they would. I checked all eleven settings; only three are universal:

random_page_cost=1.1 and effective_io_concurrency=200 — those are facts about SSDs, not about machine size
a TTL on the system logs — unbounded growth is bad on any hardware

The other eight actively hurt a big box: background_pool_size=4 caps merge parallelism and invites "too many parts", a shrunken mark_cache_size adds disk reads, shared_buffers=64MB is absurd at 10 GB, and so on. So the config ended up in two layers — a base compose file with only the universal settings, and an opt-in overlay for constrained machines:

docker compose -f docker-compose.yml -f docker-compose.small.yml up -d

The rule I took away: if a setting is proportional to resources, it doesn't belong in your default config. Defaults should only hold what's true on 2 cores and on 20.

Takeaway

None of this is really about my project — it's about stock database defaults. ClickHouse out of the box assumes it has many cores, plenty of RAM, and that nobody's counting disk. When that isn't true, you can hand back a couple of gigabytes of disk and two-thirds of your memory with one config file. Just test it locally first.

The tool I was measuring is github.com/OtezVikentiy/gotcha — Go, self-hosted, speaks the Sentry SDK protocol and OTLP. Everything above is reproducible: the configs are in the repo and the box is described up top.