DEV Community

Cover image for Kiponos vs GrowthBook — Warehouse Experiments vs Operational Config Trees (Architecture)
Moshe Avdiel
Moshe Avdiel

Posted on • Originally published at github.com

Kiponos vs GrowthBook — Warehouse Experiments vs Operational Config Trees (Architecture)

Friday 16:41. The data platform finished wiring GrowthBook to the Snowflake warehouse — Bayesian experiment results for checkout_layout_v4, exposure events flowing from the product SDK, PMs reading credible intervals in the console. The experiment pipeline is healthy. Then the incident bridge opens: a processor degrades, card-testing velocity triples, and the authorization fleet needs failure_rate_threshold at 26, fraud/thresholds/block_score at 74, and runtime/hikari/maximum_pool_size bumped across eighteen Spring Boot pods — while the checkout experiment must keep running untouched.

The growth engineer proposes:

"GrowthBook has feature flags and remote config — add block_score as a config variable. One OSS stack, warehouse-native experiments."

The platform SRE counters:

"GrowthBook assigns which product variant a user sees and waits for warehouse rows to judge winners. Our circuit breaker does not have a userId — it is global system state every pod reads at 13k TPS. Bayesian experiment infrastructure is not the contract for incident floats."

GrowthBook is a mature open-source experimentation platform — feature flags, A/B tests, Bayesian statistics, warehouse-connected metrics, and a strong OSS story for data-driven product teams. Kiponos.io is a live operational config hub — typed nested trees, WebSocket deltas, and local reads in Java Spring Boot 3 and Python. Run GrowthBook for which users get which product variant; Kiponos for how services behave under load.

The problem — experiment variant assignment standing in for operational knobs

Typical GrowthBook integration for product experiments:

// Correct — user-bound experiment assignment
GrowthBook growthbook = new GrowthBook(context);
growthbook.setAttributes(Map.of(
        "id", customerId,
        "tenantId", tenantId,
        "plan", planTier
));

if (growthbook.isOn("checkout_layout_v4")) {
    return renderVariantB();
}
return renderControl();
Enter fullscreen mode Exit fullscreen mode

The anti-pattern appears when ops keys land in the same experiment config:

# application.yml — static until someone "fixes" it with experiment config
resilience4j:
  circuitbreaker:
    instances:
      payments:
        failureRateThreshold: 45   # needs live drop during processor outage
Enter fullscreen mode Exit fullscreen mode 
// Anti-pattern — system-bound float through experiment SDK
GrowthBookContext ctx = GrowthBookContext.builder()
        .attributes(Map.of("id", "system-circuit-payments"))
        .build();
GrowthBook gb = new GrowthBook(ctx);

int failureThreshold = gb.getFeatureValue("failure_rate_threshold", 45);

if (rollingFailureRate > failureThreshold) {
    return CircuitDecision.open();
}
Enter fullscreen mode Exit fullscreen mode

Pain points:

  • Variant assignment is not system state — block_score = 74 is global fraud policy, not a checkout layout bucket
  • Bayesian stats assume exposure events — circuit thresholds need instant global change, not cohort analysis over warehouse lag
  • Warehouse-connected metrics are for product learning — incident knobs do not need Snowflake sync to decide whether to open a breaker
  • No nested ops tree — resilience/payments/partner_stripe/failure_rate_threshold becomes flat experiment feature keys
  • Python velocity workers + Java authorization — no shared ops tree without custom sync around experiment config

GrowthBook is excellent OSS for warehouse-backed product experiments. It is the wrong shape for operational configuration trees.

What teams believe vs production reality

Belief Production reality
"GrowthBook feature flags cover all runtime config" Built for variant assignment, not nested platform ops trees
"OSS experiments are free so use one system" Postgres + warehouse pipelines have real cost; scope creep hurts
"Remote config parameters fit circuit breakers" Breakers need instant global floats, not per-user experiment buckets
"Bayesian stats justify one config plane" PM experiment analysis and SRE incident keys have different change velocity
"We will add Spring Cloud Config for the rest" Now GrowthBook + Config Server + YAML for one platform

The Aha

GrowthBook assigns product variants per user and measures outcomes through warehouse-connected Bayesian analysis. Kiponos holds operational knobs — floats, nested paths, shared cross-service state — with local reads on the hot path. Keep checkout_layout_v4 in GrowthBook with full experiment analytics. Move block_score, maximum_pool_size, and failure_rate_threshold to Kiponos.

What Kiponos.io is alongside GrowthBook experiments

Kiponos is a real-time configuration hub. SDKs connect via WebSocket, load profile ['payments']['platform']['prod']['live'], and mirror the tree in memory. Edit in dashboard → delta → next getInt() in any pod — no restart, no experiment redeploy, no refresh scope.

GrowthBook remains your OSS experimentation control plane for product. Kiponos becomes your ops config plane for values that change during incidents and capacity events — same hub for Java APIs and Python workers.

Architecture — GrowthBook product experiments vs Kiponos ops hub

Architecture diagram

GrowthBook stays on user-bound experiment paths. Kiponos serves system-bound values both runtimes share.

Config tree — ops structure GrowthBook was not designed to hold 

fraud/
  thresholds/
    block_score: 74
    review_score: 61
    velocity_per_hour: 22
    strict_mode_multiplier: 1.2
resilience/
  payments/
    failure_rate_threshold: 26
    wait_duration_open_ms: 28000
    permitted_calls_half_open: 8
  partner_stripe/
    failure_rate_threshold: 22
    slow_call_rate_threshold: 0.4
runtime/
  tomcat/
    max_threads: 240
    accept_count: 200
  hikari/
    maximum_pool_size: 42
    minimum_idle: 14
    connection_timeout_ms: 5000
ml/
  velocity/
    batch_size: 128
    worker_concurrency: 6
    scoring_model_version: v2.4.0
growthbook_bridge/
  # Document experiments that stay on GrowthBook
  checkout_layout_v4: growthbook_owned
  pricing_banner_test: growthbook_owned
Enter fullscreen mode Exit fullscreen mode

Java integration — circuits and fraud on local reads 

@Configuration
public class KiponosConfig {

    @Bean
    public Kiponos kiponos(
            @Value("${kiponos.team-id}") String teamId,
            @Value("${kiponos.access-key}") String accessKey,
            @Value("${kiponos.profile-path}") String profilePath) {
        return Kiponos.builder()
                .teamId(teamId)
                .accessKey(accessKey)
                .profilePath(profilePath)
                .build();
    }
}
Enter fullscreen mode Exit fullscreen mode 
@Service
public class PaymentsCircuitService {

    private final Kiponos kiponos;

    public PaymentsCircuitService(Kiponos kiponos) {
        this.kiponos = kiponos;
    }

    public boolean shouldTripCircuit(double rollingFailureRate) {
        int threshold = kiponos
                .path("resilience", "payments")
                .getInt("failure_rate_threshold");
        return rollingFailureRate > threshold;
    }

    public FraudDecision evaluateVelocity(int hourlyVelocity, int riskScore) {
        var fraud = kiponos.path("fraud", "thresholds");
        int velocityLimit = fraud.getInt("velocity_per_hour");
        int blockScore = fraud.getInt("block_score");

        if (hourlyVelocity > velocityLimit) {
            return FraudDecision.block("velocity_exceeded");
        }
        if (riskScore >= blockScore) {
            return FraudDecision.block("score_exceeded");
        }
        return FraudDecision.allow();
    }
}
Enter fullscreen mode Exit fullscreen mode

Resize Hikari when ops bumps pool size during a traffic spike:

@PostConstruct
void bindPoolKnobs() {
    kiponos.afterValueChanged(change -> {
        if (change.getPath().startsWith("runtime/hikari/maximum_pool_size")) {
            hikariDataSource.setMaximumPoolSize(change.getNewValueAsInt());
        }
    });
}
Enter fullscreen mode Exit fullscreen mode

Product experiment — keep GrowthBook where Bayesian variant assignment belongs:

public CheckoutLayout resolveLayout(String customerId, String planTier) {
    GrowthBook growthbook = new GrowthBook(
            GrowthBookContext.builder()
                    .attributes(Map.of("id", customerId, "plan", planTier))
                    .build());
    return growthbook.isOn("checkout_layout_v4")
            ? CheckoutLayout.VARIANT_B
            : CheckoutLayout.CONTROL;
}
Enter fullscreen mode Exit fullscreen mode

Python integration — velocity worker shares the ops tree 

import os
from kiponos import Kiponos

os.environ["KIPONOS_PROFILE"] = "['payments']['platform']['prod']['live']"
kiponos = Kiponos.create_for_current_team()


def score_velocity_batch(transactions: list[dict]) -> list[str]:
    batch_size = kiponos.path("ml", "velocity").get_int("batch_size", 128)
    block_score = kiponos.path("fraud", "thresholds").get_int("block_score", 85)
    velocity_limit = kiponos.path("fraud", "thresholds").get_int("velocity_per_hour", 20)
    # velocity scoring logic ...
    return decisions


def on_config_change(change):
    if change.path.startswith("ml/velocity/batch_size"):
        reconfigure_executor(int(change.new_value))


kiponos.after_value_changed(on_config_change)
Enter fullscreen mode Exit fullscreen mode

GrowthBook has no natural home for Python velocity workers and Java authorization services sharing fraud/thresholds/block_score with sub-second edits during a BIN attack — warehouse sync latency is for product learning, not incident response.

Real scenarios

Event GrowthBook alone GrowthBook + Kiponos
Bayesian checkout_layout_v4 test with warehouse metrics Native experiment pipeline Keep GrowthBook; unchanged
Processor outage — tighten circuit threshold Fake system user + experiment config resilience/payments/failure_rate_threshold immediate
BIN attack — lower block score Not the tool; warehouse lag irrelevant fraud/thresholds/block_score in seconds
Traffic spike — raise Hikari pool size Remote config workaround + deploy runtime/hikari/maximum_pool_size live
Python + Java aligned fraud thresholds Custom sync around experiment features One Kiponos profile, two SDKs
Tomcat thread exhaustion during flash sale Restart or static YAML runtime/tomcat/max_threads with live bind

Performance — warehouse experiments vs ops hub reads

  • GrowthBook isOn() / getFeatureValue() — attribute hashing + variant lookup — ideal for per-request product assignment
  • GrowthBook for global numeric state — wrong abstraction; exposure logging adds needless overhead on auth paths
  • Warehouse metric sync — correct for Bayesian winner selection; irrelevant latency for circuit breaker floats
  • Kiponos getInt() — pure in-memory path walk at 13k TPS on authorization hot path
  • WebSocket deltas — one key change propagates without redeploying experiment definitions or Spring pods
  • Polyglot — Java Spring Boot 3 and Python workers share one hub; GrowthBook SDKs exist but do not solve nested ops trees

Honest comparison table

Criterion GrowthBook (OSS) Kiponos Honest verdict
Open-source A/B experiments Core strength Not an experiment server GrowthBook for product tests
Bayesian stats + warehouse metrics Excellent Not analytics GrowthBook wins product learning
Self-hosted control Full data sovereignty Managed hub — evaluate policy Depends on InfoSec
Numeric ops thresholds Remote config workarounds First-class Kiponos for floats
Nested cross-service config trees Flat feature keys Hierarchical paths Kiponos for platform ops
Hot-path read at 13k TPS Variant evaluation + events Local cache Kiponos on money path
Live pool / thread tuning Not designed for this afterValueChanged binds Kiponos for JVM knobs
Java + Python same ops hub Partial Both SDKs Kiponos for polyglot ops
Per-user variant stickiness Native Application concern GrowthBook for product
Operational cost Self-host + warehouse pipelines Team/hub pricing Scope each system narrowly

When not to use Kiponos

Use case Better tool
A/B experiments with Bayesian warehouse analysis GrowthBook
Open-source experiment platform on-prem GrowthBook
Product variant assignment per user segment GrowthBook
Bootstrap secrets and DB passwords Vault / K8s Secrets
Infrastructure desired state GitOps / Terraform

Getting started (15 minutes) — keep GrowthBook for experiments only

  1. Audit GrowthBook feature catalog: mark each as product experiment vs misplaced ops knob.
  2. TeamPro at kiponos.io — profile ['payments']['platform']['prod']['live'].
  3. Migrate three ops keys off experiment config: block_score, maximum_pool_size, one failure_rate_threshold.
  4. Wire Java PaymentsCircuitService and Python velocity worker to the same profile; add Hikari bind hook.
  5. Document RFC: "GrowthBook owns warehouse-backed product experiments; Kiponos owns operational config trees."

Further reading

Kiponos.io — GrowthBook for which variant wins in the warehouse. Live hub for how hard fraud blocks and circuits trip.

Top comments (0)