When operating self-hosted or cloud infrastructure for CI/CD pipelines, one subtle configuration mistake can cause unexpected resource exhaustion or build instability.
In my recent contribution to GitLab Runner (MR !7782), I solved a long-standing developer trap related to TOML configuration parsing that previously led to runners processing unlimited parallel jobs unintentionally.
Here is the story of how a small alias mapping prevents critical operational failures in production CI/CD environments.
The Problem: The "Concurrency" Trap
GitLab CI/CD documentation and community discussions frequently use the word "concurrency" to describe how many jobs a runner or worker pool can execute simultaneously. Furthermore, within a runner's specific configuration, settings like request_concurrency already use that exact terminology.
Consequently, DevOps engineers and system administrators frequently wrote:
concurrent = 1
[[runners]]
name = "docker-runner"
url = "https://gitlab.com"
concurrency = 1 # ⚠️ Expected: Only run 1 job at a time
executor = "docker"
Or at the global level:
concurrency = 4 # ⚠️ Expected: Maximum 4 global concurrent jobs
What actually happened?
Under the hood, GitLab Runner uses BurntSushi/toml to unmarshal config.toml into Go structs. Because Go struct tags strictly mapped:
-
concurrentto globalConfig.Concurrent -
limitto runner-levelRunnerConfig.Limit
Any key named concurrency was silently ignored without raising an error or warning!
Because limit defaulted to its zero value (0), and 0 in GitLab Runner represents unlimited, a runner meant to strictly execute one build at a time would suddenly pick up and run dozens of jobs in parallel. This often resulted in Out-Of-Memory (OOM) errors, CPU throttling, and failed deployments.
The Solution: Intelligent Alias Fallback
In Merge Request !7782, we implemented alias handling during configuration loading:
-
Global Level: If
concurrentis not set butconcurrencyis present in the configuration,concurrencyis mapped directly toConfig.Concurrent. -
Runner Level (
[[runners]]): Iflimitis not explicitly set,concurrencyis mapped directly toRunnerConfig.Limit. -
100% Backward Compatibility: Canonical keys (
concurrentandlimit) take priority if both are specified, ensuring existing configurations remain unaffected.
# Now fully supported and safe!
concurrency = 4
[[runners]]
name = "production-runner"
url = "https://gitlab.com"
concurrency = 1 # Enforces single-job execution as expected
executor = "docker"
Verifying with Table-Driven Tests in Go
To ensure this fallback works reliably across all combinations of configurations, we added comprehensive table-driven tests (TestLoadConfig_ConcurrencyAlias):
-
Scenario A: Only
concurrencyspecified at root level → Maps toConcurrent. -
Scenario B: Only
concurrencyspecified in[[runners]]→ Maps toLimit. -
Scenario C: Both
concurrent/limitandconcurrencypresent → Canonical fields take precedence. - Scenario D: Unset values retain their standard defaults.
go test -v ./common -run '^TestLoadConfig_ConcurrencyAlias$'
All test cases pass cleanly, safeguarding runner stability across minor and major upgrades.
Key Takeaways
- Developer UX matters in Infra Configs: If operators commonly guess a configuration key based on natural naming conventions or documentation phrasing, supporting it as an alias eliminates dangerous failure modes.
- Beware of Silent Unmarshaling: When unmarshaling TOML/YAML/JSON in Go, unused fields don't always produce syntax errors, which can leave critical limits at their uninitialized zero values.
- Open Source Contribution: Small, well-targeted improvements can resolve issues open for years (Issue #4793) and prevent real-world production outages.
Check out the merged implementation on GitLab:
👉 GitLab Runner MR !7782
Author: Manish Kumar
GitLab: @manishpatel00

Top comments (0)