DEV Community

Alessio
Alessio

Posted on Originally published at py-forge.com

How to Find What's Bloating Your Deploy Zip (and Strip It)

#python #devops #cli #webdev

You zip a project folder to upload it to a server or a serverless function, and the archive is a lot bigger than the code you wrote. Usually the extra weight comes from files that never run in production: __pycache__ folders, .map source maps, old .log files, .git leftovers, and the same asset copied twice.

PackSlim is a small Python script that finds those files, shows how much space each kind takes up, and writes a cleaned zip, tar.gz or folder with only the remaining files. This tutorial shows what it really does, what its output looks like on a sample project, and where it can catch you out.

Install: one file, standard library only

PackSlim is a single .py file. It imports only the Python standard library (argparse, fnmatch, hashlib, zipfile, tarfile and a few others), so you don't need pip install. It uses the walrus operator, so you need Python 3.8 or newer.

Save the script as packslim.py and check the options:

python packslim.py --help
Enter fullscreen mode Exit fullscreen mode 
usage: PackSlim [-h] [-o OUTPUT] [-f {zip,tar.gz,dir}]
                [-p {standard,aggressive,python,node}] [-x EXCLUDE]
                [--no-dedup] [--dry-run] [--downloads DOWNLOADS]
                [--egress-cost EGRESS_COST] [--json] [--demo]
                [source]
Enter fullscreen mode Exit fullscreen mode

If you want to try it before pointing it at real code, python packslim.py --demo creates a packslim_demo_sandbox/ folder with fake files and packs it into packslim_demo_output.zip.

Step 1: dry run on your project

I made a small sample project: a Flask-style app with a __pycache__ file, a 586 KB source map, a server log, a duplicated minified JS file, a .github/ workflow, a .pytest_cache/ and a .env. Then I ran an analysis only, with --dry-run so nothing gets written:

python packslim.py myapp --dry-run -x ".env*"
Enter fullscreen mode Exit fullscreen mode

This is the real output (banner removed):

[*] Target Directory  : /tmp/ps/myapp
[*] Active Preset     : STANDARD
[*] Duplicate Purging : ENABLED (SHA-256)
[*] Scanning file hierarchies...

================================================================================
METRIC                                        | OPTIMIZATION VALUE
--------------------------------------------------------------------------------
Source Artifacts Scanned                      | 16 files
Original Footprint                            | 1.06 MB
Production Retained Footprint                 | 195.46 KB (8 files)
Total Dead Bloat Eliminated                   | 888.70 KB (8 files)
Redundant Duplicate Copies Pruned             | 1 duplicates (195.31 KB)
Overall Bundle Reduction Ratio                | 81.97% payload reduction
--------------------------------------------------------------------------------

=== BLOAT CATEGORY BREAKDOWN ===
Category                            | Count      | Payload Eliminated
------------------------------------------------------------------------
Build Logs & Dumps                  | 1          | 78.12 KB
VCS Artifacts                       | 2          | 15.00 B
Custom Excludes                     | 1          | 9.00 B
Caches & Bytecode                   | 2          | 29.30 KB
Source Maps & Debug Symbols         | 1          | 585.94 KB
Exact Duplicate                     | 1          | 195.31 KB
------------------------------------------------------------------------
Enter fullscreen mode Exit fullscreen mode

The breakdown answers the question "what is bloating my bundle?" by category. In this sample, the source map and the duplicate JS file account for most of the extra weight.

Below that, PackSlim prints a "financial ROI estimate". Treat it as a rough calculation, not a measurement (see the "Limitations" section).

Step 2: write the slim archive

Add -o to write the output. The format is zip by default; -f tar.gz and -f dir are the other choices.

python packslim.py myapp -x ".env*" -o build/myapp-slim.zip
Enter fullscreen mode Exit fullscreen mode 
[OK] Optimized Package Ready: /tmp/ps/build/myapp-slim.zip
[OK] Final Output Footprint: 196.33 KB
Enter fullscreen mode Exit fullscreen mode

The zip that came out contains:

main.py
requirements.txt
LICENSE
README.md
app/core.py
app/utils.py
static/app.min.js
tests/test_core.py
Enter fullscreen mode Exit fullscreen mode

Your code, the minified JS and the metadata files are kept. The cache, the log, the source map, the .git/.github leftovers, the .env (excluded with -x) and the second copy of app.min.js are left out.

How it works

The scan is a plain os.walk. For each file, the name and every part of its relative path are checked against glob groups, and the first matching group decides the category:

for grp, patterns in self.pattern_groups.items():
    if self._matches_any(f, rel_p, patterns):
        is_bloat = True
        category = grp
        reason = f"Matched filter group '{grp}'"
        break
Enter fullscreen mode Exit fullscreen mode

Matching runs against each path part, so a directory pattern like __pycache__* or .git* catches every file below that directory.

Files that don't match any pattern are hashed with SHA-256. The first copy of a file is kept, and any later file with the same content is marked "Exact Duplicate". --no-dedup turns this off.

Packaging just copies the files that aren't flagged:

keep_files = [f for f in files if not f.is_bloat]
Enter fullscreen mode Exit fullscreen mode

The zip and tar.gz outputs use compression level 9.

Presets

  • standard: VCS folders (.git*, .svn*, .hg*), Python and tool caches, *.log/*.dump, source maps and .pdb, swap and PID files.
  • aggressive: everything in standard, plus tests and fixtures, docs and all *.md, *.txt and LICENSE* files, and design files (.psd, .fig, .sketch, .ai).
  • python: Python caches, *.egg-info, venv/.venv, .env*, test caches, .vscode/.idea.
  • node: npm/yarn caches, *.tsbuildinfo, *.map, docs, test files.

Presets don't stack. -p python doesn't remove .log, .map or .git files, and standard doesn't remove .env. To fill the gaps, use -x (you can repeat it), as in -x ".env*" above.

Using it in CI

--json prints the same statistics as structured data. The exit code is 0 whether or not bloat was found, so if you want a build to fail, check the report yourself. For example, to fail when the source tree contains a source map:

python packslim.py myapp --dry-run --json -x ".env*" > packslim-report.json
python - <<'PY'
import json, sys
report = json.load(open("packslim-report.json"))
breakdown = report["statistics"]["bloat_breakdown"]
maps = breakdown.get("Source Maps & Debug Symbols", {}).get("count", 0)
if maps:
    sys.exit(f"{maps} source map(s) found in the source tree")
print("bundle clean")
PY
Enter fullscreen mode Exit fullscreen mode

On the sample project, this printed 1 source map(s) found in the source tree and exited with status 1.

Limitations (read before shipping)

  • The output must be outside the source folder. PackSlim refuses (exit code 2) if -o is the source folder, sits inside it, or is a parent of it. With -f dir, an existing output folder is replaced.
  • No per-file report and no "keep" list. The dashboard and the JSON only show counts by category. To see exactly what was removed, write with -f dir to a new folder and diff it against the source.
  • aggressive removes files you may need: requirements.txt (every *.txt), LICENSE, README.md, and any path part starting with test.
  • Duplicates are removed, not linked. If your code loads both copies by path, the second path will be missing.
  • The ROI numbers are an estimate. The egress figure is removed bytes × --downloads × --egress-cost. CI build count (1200/month) and network speed (120 Mbps) are fixed in the code. Set the two CLI values to your own numbers, or ignore that section.
  • .gitignore and .github/ count as "VCS Artifacts", which is fine for a deploy bundle but surprising the first time.

FAQ

Does it change my project folder?
No. The source is only read. The output has to be a path outside the source folder; with -f dir, an existing output folder is replaced.

Does it shrink the code itself, like minifying?
No. It works on whole files: each one is either kept or left out. It doesn't edit file contents.

Can I use it on a JavaScript project?
Yes. Matching is by file name, so -p node (or standard for .map and log files) works on any folder. It doesn't read package.json or resolve imports.

What does --demo do?
It creates ./packslim_demo_sandbox with sample files and writes ./packslim_demo_output.zip in the current folder.

Full version and the rest of the toolkit: PackSlim on PyForge. Product page: https://py-forge.com/products/packslim

Or get all 23 tools with the All-Access Pass for $99.

Top comments (0)