How floci-gcp builds GCP wire compatibility on Google's own protobuf (and stays maintainable)

If you've ever tried to test a Google Cloud app on your laptop, you know the drill. You want Pub/Sub, so you start the Pub/Sub emulator on :8085. You also use Firestore, so that's a second emulator on :8080. Datastore needs its own. Cloud Storage has no official emulator at all, so you reach for a third-party one on :4443. Secret Manager, IAM, Cloud Tasks? Now you're stubbing those by hand.

That's up to seven services, seven processes, seven ports, and seven little setup rituals, each one a separate binary with its own flags and its own lifecycle.

floci-gcp folds all of that into one container on one port:

docker run -d -p 4588:4588 floci/floci-gcp:latest

export PUBSUB_EMULATOR_HOST=localhost:4588
export FIRESTORE_EMULATOR_HOST=localhost:4588
export DATASTORE_EMULATOR_HOST=localhost:4588
export STORAGE_EMULATOR_HOST=http://localhost:4588
export SECRET_MANAGER_EMULATOR_HOST=localhost:4588
export GOOGLE_CLOUD_PROJECT=floci-local

Your existing SDK code works without a single change. No account, no service-account key, no auth token, no feature gates.

But the convenience isn't really what I want to write about. The thing worth talking about is the decision underneath it, the one that lets a small project get the wire format exactly right without sinking under maintenance. For GCP in particular, there's a shortcut available that simply doesn't exist on AWS.

The contract problem

Every cloud emulator has the same core job: convince a real SDK that it's talking to the real cloud. The SDK serializes a request, sends bytes over the wire, and expects bytes back in exactly the shape it knows how to read. Get one field wrong and the client throws.

So the real question for any emulator is this. Where does the wire contract actually come from?

For AWS, the honest answer is "nowhere public, exactly." AWS doesn't publish a complete, machine-readable spec of every byte on the wire. Emulating it means treating the SDK's own deserializer source as the de facto contract and reverse-engineering the shapes it will accept. That works, but it's archaeology, and you do it all over again every time a service grows a new field.

GCP turns out to be the opposite, and that flip is the whole insight. Google publishes the contract as protobuf. The .proto files that define Pub/Sub, Firestore, Datastore, Secret Manager, and Cloud Tasks are public, versioned, and compiled into officially released gRPC stub artifacts on Maven Central. The wire format isn't a secret you have to recover. It's a dependency you download.

floci-gcp leans on that all the way.

Implementing the server side of Google's own stubs

Here's the move. Google's published artifacts don't just hand you the message types. They also include the generated service base classes, the abstract server-side handlers (the ...ImplBase classes) that you'd normally subclass to build a real gRPC server. floci-gcp subclasses them directly:

public class FirestoreController extends FirestoreGrpc.FirestoreImplBase {

    @Override
    public void commit(CommitRequest request,
                       StreamObserver<CommitResponse> responseObserver) {
        // request is com.google.firestore.v1.CommitRequest,
        // Google's generated type, not ours
        ...
    }
}

CommitRequest, CommitResponse, Write, WriteResult, every one of those is imported from com.google.firestore.v1.*, straight out of Google's published artifact. floci-gcp never declares them. The same pattern runs across all of the gRPC services:

Service	Base class implemented	Source artifact
Pub/Sub	PublisherGrpc.PublisherImplBase, SubscriberGrpc.SubscriberImplBase	grpc-google-cloud-pubsub-v1
Firestore	FirestoreGrpc.FirestoreImplBase	grpc-google-cloud-firestore-v1
Secret Manager	SecretManagerServiceGrpc.SecretManagerServiceImplBase	grpc-google-cloud-secretmanager-v1
Cloud Tasks	CloudTasksGrpc.CloudTasksImplBase	grpc-google-cloud-tasks-v2
Datastore	proto types from proto-google-cloud-datastore-v1	binary protobuf over HTTP

The consequence is the part worth sitting with. floci-gcp does not model the GCP wire format. The message types inside floci-gcp are byte-identical to production for the simplest possible reason: they are Google's generated types. There's no second set of POJOs to keep in sync with reality, because there's no second set at all. The only code in the repo is the behavior. What a commit actually does, where the bytes get stored, what comes back.

That's the point where wire fidelity stops being something you chase and starts being a structural property. For every RPC floci-gcp implements, the bytes match because the compiler makes them match. I'm not going to pretend it covers all of GCP yet, because it doesn't. There are services and edge cases still to fill in, and that's exactly the work in front of us. But the part that's usually the hard, fiddly grind, getting the format right, is the part this approach hands you for free. What's left is breadth, not accuracy.

Why it stays maintainable

The maintenance story falls out of the very same decision.

When Google evolves a service, say it adds a field to a request, a new RPC, or a new enum value, that change lands in their published proto, which lands in the next version of the stub artifact. Inside floci-gcp, absorbing it is a dependency version bump. You're not hunting through hand-written types trying to mirror the change, because you never wrote those types in the first place.

The project bakes this into a hard rule for contributors: consume the precompiled stubs, never check in raw .proto codegen. The contributor guide spells it out plainly. Don't introduce custom endpoint shapes, don't reshape requests or responses for convenience, and match what the SDK and the gcloud CLI actually do. The protobuf dependency is the spec, and drifting from it is the one thing you're not allowed to do.

That's the difference between an emulator that's compatible right up until the next SDK release and one whose compatibility is anchored to the same source of truth the SDK itself compiles against.

One port, gRPC and REST together

There's a second piece that makes the single-port claim more than cosmetic. GCP services don't all speak the same protocol. Pub/Sub, Firestore, Secret Manager, and Cloud Tasks are protobuf over gRPC. Datastore is binary protobuf over plain HTTP. Cloud Storage and IAM are REST, and GCS in particular mixes REST JSON for bucket management with REST XML for object operations.

Normally that's a reason to run several listeners side by side. floci-gcp serves all of it on a single HTTP/2 port using ALPN negotiation. It's built on Quarkus, and two lines of config do the heavy lifting:

quarkus.http.http2=true
quarkus.grpc.server.use-separate-server=false

Setting use-separate-server=false tells the gRPC layer not to stand up its own listener, so it shares the main HTTP server instead. ALPN then picks the right protocol per connection. An h2 client gets gRPC, an HTTP/1.1 client gets REST, all on the same :4588.

The whole thing ships as a GraalVM native image, so it starts in milliseconds and idles on very little memory. That's what makes it realistic to spin up a fresh instance inside a CI job instead of babysitting a long-lived fleet of emulators.

Where this sits in the landscape

It's worth being clear about prior art, because the existing GCP emulators are genuinely good at what they do.

Tool	Scope	Ports	Notes
Official gcloud / Firebase emulators	Pub/Sub, Firestore, Datastore, Bigtable, Spanner	one per service	Built by Google, high fidelity. But fragmented, with a separate binary, port, and setup for each. No GCS, Secret Manager, IAM, or Tasks emulator.
fake-gcs-server (fsouza)	Cloud Storage only	:4443	Mature, widely used, the go-to way to test GCS. Single service by design.
floci-gcp	GCS, Pub/Sub, Firestore, Datastore, Secret Manager, IAM, Managed Kafka, plus Cloud Tasks	one (:4588)	gRPC and REST on one port, Google's own protos, native image.

The official emulators win on fidelity for the services they cover, since they're built by the team that owns the API. fake-gcs-server is the respected incumbent for GCS and isn't going anywhere. floci-gcp is making a different bet: unify the common set behind one endpoint, and keep fidelity high by borrowing Google's published contract rather than approximating it.

If your test suite touches a single GCP service, the single-purpose tools are great. Once it touches four, the math starts to change.

Try it

# docker-compose.yml
services:
  floci-gcp:
    image: floci/floci-gcp:latest
    ports:
      - "4588:4588"

import os
os.environ["FIRESTORE_EMULATOR_HOST"] = "localhost:4588"

from google.cloud import firestore

db = firestore.Client(project="floci-local")
db.collection("users").add({"name": "Alice", "age": 30})
docs = db.collection("users").where("name", "==", "Alice").stream()
for doc in docs:
    print(doc.to_dict())

If you care about durability across restarts, pick a storage mode through FLOCI_GCP_STORAGE_MODE. The options are memory (the default, fastest, ideal for CI), persistent, hybrid, and wal.

Part of a family

floci-gcp is the Google Cloud member of Floci, a family of free, MIT-licensed local cloud emulators. The AWS emulator covers 45 services, and there's an Azure one too. All three share the same philosophy: real wire protocols instead of mocks, a single endpoint, a native image, and never gating an emulated service behind payment.

The fun footnote is that the GCP edition got to lean on a trick the AWS edition can't. On AWS, the SDK source is the contract and you reverse-engineer it. On GCP, Google ships the contract as protobuf and you compile against it. Same goal, much shorter path.

It's all open source, and the code is worth a read if this kind of thing is your idea of a good time: github.com/floci-io/floci-gcp.

---

I maintain Floci. I'm always happy to hear what's working, what's broken, and which service you'd want to see next. Issues and PRs welcome.