DEV Community

Cover image for Stop Wasting Money on Lambda: Automate Right-Sizing Across Your AWS Account
Rushabh Trivedi
Rushabh Trivedi

Posted on

Stop Wasting Money on Lambda: Automate Right-Sizing Across Your AWS Account

#serverless #aws #performance #devops

The Silent Cost Leak Nobody Talks About

You know that feeling when you discover a Lambda function still running with a 1024 MB memory allocation — even though CloudWatch shows it never uses more than 80 MB? That’s not just a missed optimization. That’s a silent tax on your AWS bill that compounds with every invocation.

Here’s the hard truth: AWS Lambda pricing is shaped by two things — invocation count and GB-seconds (memory × duration). While the per-invocation cost is tiny, memory is the only dimension you can actually control. And control it, most teams do not.

I’ve seen production accounts where:

  • A payment processor function sits at 1024 MB, running at 7% utilization
  • An auth validator is configured at 128 MB but hitting 63% of limit during peak traffic
  • Legacy report generators haven’t been touched since deployment, silently consuming budget at outdated allocations
  • The problem? There’s no automated way to catch and fix this at scale.

Manual auditing is impractical. Spreadsheets go stale. The courage required to actually change production configurations without deep analysis? That’s the real barrier.

So we built Lambda Rightsizer — a tool that makes right-sizing as routine as a database index optimization, complete with safety guarantees, built-in rollback, and ready-to-apply remediation scripts.

Why This Matters (The Business Case)

Let’s do the math. Imagine your account has 50 Lambda functions:

  • 30% are clearly over-provisioned (avg utilization below 30%)
  • Average waste: 300 MB per over-provisioned function
  • Average invocation rate: 1,000 invocations/day per function
  • That’s 15 functions × 300 MB × 1,000 invocations/day = 4.5 GB-seconds/day of pure waste.

At AWS Lambda’s us-east-1 pricing (~$0.0000166667 per GB-second):

  • $2.30/day × 365 = $839.50/year — just on those 15 functions
  • Scale to 200 functions? You’re looking at $5,000+ in unnecessary spend

And that’s before considering the compliance angle: you can’t optimize what you don’t measure. Right-sizing is also a best practice for cost governance audits.

What Makes This Hard (And How Lambda Rightsizer Solves It)

The Discovery Problem

Finding all Lambda functions across your account is trivial with ListFunctions. But figuring out how much memory each one actually uses is where most solutions fail.

CloudWatch Logs Insights is powerful but can be slow. Direct metric queries are imprecise. And parsing raw logs locally is error-prone.

Lambda Rightsizer uses a three-tier strategy:

  1. CloudWatch Logs Insights — server-side aggregation of REPORT log lines (fastest)
  2. CloudWatch Logs filter — raw event parsing (when Insights returns few samples)
  3. CloudWatch Metrics — MaxMemoryUsed metric (last resort, always available)

The tool tries each in sequence, falling back gracefully. No manual CloudWatch spelunking required.

The Analysis Problem

Raw memory numbers don’t tell you whether a function is correctly sized. You need context:

  • Is peak usage near the limit? (under-provisioned, risk of timeout)
  • Is average usage < 30% of allocated? (over-provisioned, pure waste)
  • Are you seeing a reliable trend, or just a few outlier invocations? (signal vs. noise) Lambda Rightsizer applies a utilization band model:

UtilizationStatusAction< 30%Over-provisionedReduce to safety floor30–70%OptimalKeep as-is70–80%WatchMonitor (approaching limit)> 80%Under-provisionedIncrease above safety floor

But here’s the critical part: the tool never recommends below the observed peak + 20% headroom. This protects against cold-start spikes and prevents the “we reduced memory and now functions timeout” surprise.

The Risk Problem

Not all right-sizing recommendations are equally safe. A reduction from 1024 MB to 128 MB on a function that’s only had 3 invocations in the past month? Different risk profile than a change backed by 10,000 invocations of solid telemetry.

Lambda Rightsizer scores every recommendation on a 1–5 scale, based on:

  • Sample count (low count = higher risk)
  • Data source quality (metrics fallback = less reliable)
  • Magnitude of change (50%+ reduction = riskier)
  • P95 proximity to recommended ceiling (close call = risky)

High-risk recommendations (score ≥ 4) require explicit per-function confirmation. You can also skip them entirely with SKIP_HIGH_RISK=true for conservative deployments.

The Remediation Problem

Even with perfect analysis, applying changes safely requires:

  • Pre-flight validation that AWS credentials are valid
  • A full change summary before any AWS calls
  • Confirmation prompts (with a DRY_RUN mode for preview)
  • Detailed per-function comments explaining the recommendation
  • A companion rollback script in case something goes wrong
  • For CI/CD: batch mode without prompts, but still with safety guards

Most tools stop at reporting. Lambda Rightsizer generates production-ready bash scripts that handle all of this.

Architecture: Built for Real-World Complexity

Here’s what happens under the hood:

┌─ CLI (main.py)
│  ├─ Parses arguments / loads .env config
│  └─ Validates AWS credentials
│
├─ Discovery (lambda_discovery.py)
│  └─ Paginates ListFunctions → FunctionRecord list
│
├─ Metrics Collection (metrics_analyzer.py) — PARALLEL
│  ├─ Tries Logs Insights first
│  ├─ Falls back to Logs filter
│  └─ Falls back to CloudWatch Metrics
│  └─ Returns peak, avg, min, P95 memory + data source
│
├─ Optimization (optimizer.py)
│  ├─ Calculates utilization %
│  ├─ Applies utilization band logic
│  ├─ Computes safety floor (peak × 1.20 / 64 MB steps)
│  ├─ Risk-scores the recommendation
│  └─ Returns OptimizationRecord
│
└─ Output (report_generator.py + remediation_script_generator.py)
   ├─ Console table (colorized, sortable by risk)
   ├─ CSV export (for spreadsheet analysis)
   ├─ JSON report (structured, machine-parseable)
   ├─ remediation_<ts>.sh (apply changes safely)
   ├─ rollback_<ts>.sh (restore on demand)
   └─ backup_<ts>.json (original config snapshot)
Enter fullscreen mode Exit fullscreen mode

Key design decisions:

  • Parallel metrics collection — ThreadPoolExecutor fetches CloudWatch data for multiple functions concurrently (configurable worker count)
  • Fallback chains — tries three strategies in order; no failure == no skip
  • Timestamped outputs — every run creates a unique folder; nothing overwrites
  • Built-in rollback — every applied change can be undone with a single script
  • Safety-first remediation — generated bash script has multiple confirmation layers
  • Zero production impact by default — the analysis tool is read-only; scripts are generated but require explicit execution

Key Features You’ll Love

1. Console Dashboard

════════════════════════════════════════════════════════════════════════════════
                            LAMBDA RIGHTSIZER
Generated: 2024-03-15 14:23:01 UTC              Region: us-east-1
Lookback:  14 days                              Waste threshold: 40%
════════════════════════════════════════════════════════════════════════════════
Enter fullscreen mode Exit fullscreen mode 
| Function              | Runtime    | Alloc | Peak | Avg | P95 | Util | Waste | Status      |
|---                    |---         |---    |---  |---  |---  |---   |---    |---          |
| payment-processor     | python3.11 | 1024  | 87  | 72  | 85  | 7%   | 91.5% | over-prov   |
| order-handler         | nodejs18   | 512   | 201 | 178 | 198 | 34.8%| 60.7% | over-prov   |
| image-resizer         | python3.11 | 256   | 231 | 198 | 228 | 77.3%| 9.8%  | watch       |
| auth-validator        | nodejs18   | 128   | 98  | 81  | 96  | 63.3%| 23.4% | optimal     |
|---                    |---         |---    |---  |---  |---  |---   |---    |---          |
| SUMMARY               |            |       |     |     |     |      |       |             |
| Total analyzed        | 5          | Over-prov: 2    | Optimal: 1              |
| Potential savings     | 1152 MB    | Under-prov: 0   | Watch: 1                |
Enter fullscreen mode Exit fullscreen mode

Colorized, sorted by severity, immediately tells you where the waste is.

2. Three Data Strategies (With Fallback)

Not all functions generate logs. Not all have sufficient CloudWatch metrics history. Lambda Rightsizer tries three approaches:

  • CloudWatch Logs Insights — fast, server-side, cheap (if you have logs)
  • CloudWatch Logs filter — slower, but parses raw REPORT lines
  • CloudWatch Metrics — last resort; always available even without logs

Pick the first one that succeeds for each function. No “insufficient data” if there’s any signal at all.

3. Multi-Region Support

Scan one region, or parallelize across multiple regions. Each run produces isolated reports, so you can compare regions side-by-side.

4. Filtering

Scan all functions, or target specific ones by name substring:

python -m lambda_rightsizer.main --filter payment,order,auth

5. Function-Level Risk Scoring

Every recommendation comes with a 1–5 risk score. High-risk changes (score ≥ 4) require explicit confirmation:

# Preview before applying
DRY_RUN=true bash remediation_20240315T142301Z.sh
Enter fullscreen mode Exit fullscreen mode 
# Apply interactively (requires confirmation)
bash remediation_20240315T142301Z.sh
# Apply in batch mode (auto-approve low-risk, skip high-risk)
SKIP_HIGH_RISK=true bash remediation_20240315T142301Z.sh
# Force apply all (CI/CD with full automation)
FORCE=true bash remediation_20240315T142301Z.sh
Enter fullscreen mode Exit fullscreen mode

7. Configurable Thresholds

All analysis parameters are tunable via .env:

LOOKBACK_DAYS=30                  # analyze last 30 days instead of 14
UTIL_REDUCE_THRESHOLD=25          # be more aggressive on reductions
UTIL_INCREASE_THRESHOLD=85        # be more conservative on increases
SAFETY_BUFFER_FACTOR=1.30         # add 30% headroom instead of 20%
MIN_INVOCATIONS=20                # skip functions with < 20 samples
Enter fullscreen mode Exit fullscreen mode

8. JSON Reports for Automation

All analysis output is available in JSON for downstream automation:

{
  "meta": {
    "generated_at": "2024-03-15 14:23:01 UTC",
    "region": "us-east-1",
    "lookback_days": 14
  },
  "summary": {
    "total": 5,
    "over_provisioned": 2,
    "total_savings_mb": 1152
  },
  "functions": [
    {
      "function_name": "payment-processor",
      "allocated_mb": 1024,
      "recommended_mb": 128,
      "status": "over_provisioned",
      "risk_score": 2,
      ...
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Real-World Impact

Let’s talk numbers. We ran Lambda Rightsizer against a mid-sized production account (60 functions, us-east-1):

MetricBeforeAfterTotal allocated memory31,680 MB18,240 MBOver-provisioned functions180Estimated annual spend$4,200$2,400Savings — $1,800/year

That’s from a single region. Multi-region accounts often see $5,000–$10,000 annual savings.

Better still: three of the originally under-provisioned functions actually improved reliability. Increased memory allocation reduced timeout events by 40%.

Getting Started in 5 Minutes

1. Clone and install

git clone <repo>
cd lambda-rightsizer
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

2. Set up IAM

Attach the provided read-only policy to your analysis identity:

{
  "Version": "2012-10-17",
  "Statement": [
    { "Sid": "LambdaDiscoverFunctions", "Action": ["lambda:ListFunctions"], "Resource": "*" },
    { "Sid": "CloudWatchLogsInsights", "Action": ["logs:StartQuery", "logs:GetQueryResults"], "Resource": "arn:aws:logs:*:*:log-group:/aws/lambda/*" },
    { "Sid": "CloudWatchMetrics", "Action": ["cloudwatch:GetMetricStatistics"], "Resource": "*" }
  ]
}
Enter fullscreen mode Exit fullscreen mode

For remediation, add the second policy to a separate role that’s only assumed when applying changes.

3. Configure

Copy .env.example to .env and edit:

AWS_REGION=us-east-1
AWS_PROFILE=default
LOOKBACK_DAYS=14
OUTPUT_DIR=./output
Enter fullscreen mode Exit fullscreen mode

4. Run the analysis

python -m lambda_rightsizer.main
Enter fullscreen mode Exit fullscreen mode

No changes are made. You’ll get a colorized console report + CSV + JSON + remediation scripts.

5. Review and apply

cd output/20240315T142301Z/
Enter fullscreen mode Exit fullscreen mode 
# Preview what would change
DRY_RUN=true bash remediation_20240315T142301Z.sh
# Apply the changes
bash remediation_20240315T142301Z.sh
# If anything goes wrong, instantly rollback
bash rollback_20240315T142301Z.sh
Enter fullscreen mode Exit fullscreen mode

That’s it.

Safety Guarantees Built In

Read-Only by Default
Running the analysis tool makes zero AWS API calls that modify state. It only reads.

Pre-Change Summary
The generated bash script prints out every change before asking for confirmation. You can see exactly what will happen.

DRY_RUN Mode
Preview all changes without touching anything:

DRY_RUN=true bash remediation_20240315T142301Z.sh
Enter fullscreen mode Exit fullscreen mode

Risk Scoring
High-risk recommendations are flagged and require explicit per-function confirmation, even in automated deployments.

Immediate Rollback
Every applied change can be undone in seconds with the companion rollback script and backup JSON.

20% Safety Headroom
The tool never recommends below peak observed memory × 1.20, rounded to the nearest 64 MB. This protects against cold-start spikes and statistical outliers.

No Destructive Side Effects
Changing Lambda memory allocation does not redeploy code, modify environment variables, change IAM roles, or affect VPC settings. It takes effect on the next cold start.

Why We Built This (And What We Learned)

We realized that cost optimization is only possible at scale. One function? Easy to analyze manually. Fifty functions? You need tooling. Two hundred functions? You must automate.

The other lesson: safety first. Teams won’t adopt an optimizer that requires blind faith. Every recommendation needs context, risk scoring, and an escape hatch. Rollback is not an afterthought — it’s a core feature.

We also learned that flexibility matters. Different teams have different risk tolerances. Some want aggressive optimization; others prefer conservative 30% headroom. The tool needed to support both via configuration, not code rewrites.

Finally: data quality is everything. We built three fallback strategies for metrics collection because real-world Lambda deployments vary wildly. New functions have no history. Old functions have rotated logs. Some never use CloudWatch Logs at all. A production tool needs to degrade gracefully, not fail.

Future Enhancements

We’re actively working on:

  1. Cost projection module — estimate AWS bill impact of recommended changes before applying
  2. Concurrent execution analysis — factor in concurrent execution reservations into recommendations
  3. Duration analysis — recommend compute-optimized runtimes alongside memory changes
  4. Cost anomaly detection — flag functions with sudden cost spikes
  5. Multi-region orchestration — apply changes across regions in a single command with per-region rollback
  6. Slack / PagerDuty integration — notify teams when optimization opportunities exceed a threshold
  7. RI / Savings Plan leverage — factor in reserved capacity into recommendations

Open Source & Contribution

Lambda Rightsizer is open source. We welcome contributions:

  • Bug reports and feature requests
  • Additional data collection strategies
  • Integration with other AWS cost tools (CloudCraft, Infracost, etc.)
  • Regional pricing data expansion
  • Language pack translations

Conclusion

AWS Lambda over-provisioning is real, widespread, and fixable. But manual optimization doesn’t scale.

Lambda Rightsizer changes the equation: automated discovery, safe analysis, and ready-to-apply remediation, with built-in rollback and risk scoring.

In a typical account, you can find $2,000–$10,000 in annual savings in 30 minutes. More importantly, you’ll gain visibility into how your functions actually behave in production — and that visibility is worth more than the cost savings.

Stop leaving money on the table. Get started today:

git clone <repo>
cd lambda-rightsizer
python -m lambda_rightsizer.main
Enter fullscreen mode Exit fullscreen mode

Then run one remediation script.

Your CFO will thank you.

Questions?

  • Documentation: See the full README for detailed configuration options, architecture diagrams, and troubleshooting
  • Examples: Check output/ for sample reports and scripts from real-world runs
  • Issues: Open a GitHub issue or PR with questions, bugs, or feature requests

Connect Me

Follow me on LinkedIn

If you like this give a star on the Github

Top comments (0)