WonderLab

Posted on May 18

One Open Source Project a Day (No. 68): CLI-Anything - Making Every Piece of Software Agent-Native

#ai #claude #cli #opensource

Introduction

"Making ALL Software Agent-Native."

This is the 68th article in the "One Open Source Project a Day" series. Today, we are exploring CLI-Anything.

Also from the HKUDS team at the University of Hong Kong, this time they are solving an even more foundational problem: how does an AI agent control software that has no API?

Your AI agent can call GitHub APIs, query databases, send HTTP requests—but what happens when you ask it to open GIMP and process an image? Or have Blender render a 3D scene? Or use LibreOffice Calc to generate a report? These applications weren't designed with AI in mind. They only have a GUI.

CLI-Anything's answer: use a 7-phase automated methodology to wrap any software into a structured CLI—with deterministic JSON output, complete test coverage, and an auto-generated SKILL.md so AI agents can "read" how to use it. 35.7k Stars, 3.5k Forks, 80+ supported apps—the second major open-source release from the same team.

What You Will Learn

The complete 7-phase methodology (Analyze → Design → Implement → Plan Tests → Write Tests → Document → Publish)
The design philosophy behind HARNESS.md and SKILL.md—letting AI agents "read" any software
How CLI-Hub package manager builds a shareable CLI ecosystem
Why "deterministic CLI output" is better suited for AI agents than "GUI simulation"
How to generate a complete CLI wrapper for GIMP or Blender with a single command in Claude Code

Prerequisites

Basic command-line familiarity (what a CLI is)
Python environment (pip install)
Experience with Claude Code or other AI agent tools is helpful for understanding the use cases

Project Background

Project Introduction

CLI-Anything is an open-source framework that makes "all software agent-native." Its core capability: given the path to any desktop or web application, it automatically analyzes the software's features, generates a structured CLI wrapper, and lets AI agents control that software as if calling a function.

This takes a completely different technical approach from UI-TARS-Desktop (No. 98, which uses visual perception and mouse simulation):

UI-TARS approach: Read screenshot → determine where to click → simulate mouse click (visual understanding, adaptive, but non-deterministic)
CLI-Anything approach: Analyze software → generate CLI → JSON output (structured, deterministic, testable)

Both approaches have their place. CLI-Anything is better suited for automation workflows that require reliable, repeatable execution.

Author/Team Introduction

Team: HKUDS (HKU Data Science Lab, University of Hong Kong)
Same team behind: OpenHarness (No. 96) — AI agent infrastructure framework
Academic + engineering: HKUDS projects consistently combine academic rigor with engineering pragmatism
Active community: Feishu and WeChat developer communities, contributor templates via GitHub Issues, daily changelog updates

Project Data

⭐ GitHub Stars: 35,700+
🍴 Forks: 3,500+
🧪 Test Coverage: 2,600+ passing test cases (across all harnesses)
🔧 Supported Applications: 80+
📄 License: Apache-2.0
🌐 Repository: HKUDS/CLI-Anything

Main Features

Core Utility

CLI-Anything does one thing in one sentence: turns "GUI-only" software into programmatically callable tools for AI agents.

It solves the "last mile" problem between AI agents and traditional software:

AI agent capability boundary (without CLI-Anything):
  ✅ Call REST APIs
  ✅ Execute SQL queries
  ✅ Run shell commands
  ✅ Read and write files
  ❌ Control GIMP to process images
  ❌ Have Blender render scenes
  ❌ Use LibreOffice to generate reports
  ❌ Invoke FreeCAD's CAD features

AI agent capability boundary (with CLI-Anything):
  All of the above ✅

Use Cases

Creative Software Automation
- Have an AI agent batch-process images (GIMP crop + color correct + export)—complete in one command what would otherwise require dozens of manual operations.
3D Content Production Pipeline
- AI agent calls Blender CLI, from script to render output—the entire 3D content production workflow fully automated.
Bulk Document Generation
- LibreOffice's CLI wrapper lets AI agents bulk-generate, populate, and format office documents.
Game Development Automation
- Godot engine's CLI wrapper lets AI agents participate in game asset management and build pipelines.
Specialized Domain Tool Integration
- FreeCAD (mechanical CAD), molecular modeling software, GIS mapping tools—specialized applications that could never be accessed via API are now callable by AI agents.

Quick Start

In Claude Code (Fastest Path):

# 1. Install the plugin
/plugin marketplace add HKUDS/CLI-Anything
/plugin install cli-anything

# 2. Generate CLI for your target software (GIMP example)
/cli-anything ./gimp

# CLI-Anything automatically runs the 7-phase pipeline (~2–5 minutes)
# Generated file structure:
# gimp-harness/
# ├── gimp_cli.py      ← Main CLI file
# ├── HARNESS.md       ← Detailed usage guide (for humans)
# ├── SKILL.md         ← AI-agent-readable skill spec
# ├── TEST.md          ← Test plan and results
# ├── tests/           ← Unit tests + E2E tests
# └── setup.py         ← Installation config

# 3. Use immediately after generation
gimp-cli image resize --input photo.jpg --width 800 --height 600 --output resized.jpg
gimp-cli image convert --input photo.png --format jpg --quality 85 --output photo.jpg
gimp-cli batch process --input-dir ./raw/ --output-dir ./processed/ --operations "resize,sharpen"

Install Existing Harnesses via CLI-Hub:

# Install CLI-Hub
pip install cli-anything-hub

# Browse available harnesses
cli-hub list

# Install harnesses for specific apps (community-contributed versions available)
cli-hub install gimp
cli-hub install blender
cli-hub install libreoffice
cli-hub install godot
cli-hub install freecad

# Search
cli-hub search "3d modeling"

# Update all installed harnesses
cli-hub update --all

Extend an Existing Harness (Add New Capabilities):

# In Claude Code, refine an existing GIMP harness
/cli-anything:refine ./gimp-harness "Add batch watermarking and EXIF data processing"

Use in Other AI Agents:

# Pi Coding Agent
bash .pi-extension/cli-anything/install.sh
# Then: @cli-anything ./blender

# OpenCode
opencode cli-anything ./inkscape

# Manual Python development
pip install cli-anything
python -m cli_anything generate ./target-app

Core Characteristics (7-Phase Methodology)

The heart of CLI-Anything—a complete automated pipeline that transforms "software" into "CLI":

Phase 1 — Analyze

Input: Target software path ./gimp
      ↓
Automatic scanning:
  - Executable structure
  - Command-line arguments (--help output)
  - Python/scripting API (if any)
  - Plugin system (if any)
  - Documentation (man pages, README)
      ↓
Output: Feature map (GUI operations → programmable interfaces)

Phase 2 — Design

Input: Feature map
      ↓
Architecture design:
  - Command groupings (gimp-cli image / gimp-cli batch / gimp-cli script)
  - State model (session management, REPL mode)
  - Output format (JSON schema definitions)
  - Error handling specification
      ↓
Output: CLI architecture design document

Phase 3 — Implement

Built on Click framework (Python):
  - Main CLI command groups
  - Subcommands and parameters
  - JSON output formatting
  - REPL interactive mode (with undo/redo)
  - Session state management
  - Error handling

Phase 4 — Plan Tests

Generates TEST.md containing:
  - Unit test strategy (parameter boundaries for each command)
  - E2E test scenarios (complete workflow testing)
  - Exception scenario testing (invalid input, missing files, etc.)
  - Performance test baselines

Phase 5 — Write Tests

Auto-generates pytest test suite:
  - tests/unit/     ← Unit tests
  - tests/e2e/      ← End-to-end tests
  - tests/fixtures/ ← Test data

Target: 100% critical path coverage

Phase 6 — Document

Two types of documentation are auto-generated, serving two different audiences:

HARNESS.md (for humans):
  - Complete usage guide
  - All commands and parameter descriptions
  - Example workflows
  - Troubleshooting guide

SKILL.md (for AI agents):
  ---
  name: gimp-cli
  description: GIMP image manipulation CLI for AI agents
  triggers:
    - "process image"
    - "resize photo"
    - "batch images"
  commands:
    - name: image resize
      description: Resize an image to specified dimensions
      parameters:
        - name: --input
          type: file_path
          required: true
        - name: --width
          type: integer
        ...
  output_format: json
  ---

SKILL.md is the project's most elegant design—it defines a standard format that lets AI agents automatically discover and use any CLI tool without any manual configuration.

Phase 7 — Publish

Generate setup.py → pip install -e . → Install to system PATH
→ Tool is immediately available: $ gimp-cli --help
→ Register to CLI-Hub (optional, for community sharing)

Supported Applications (80+, Selected)

Category	Representative Apps
Image Processing	GIMP, Inkscape, Krita, ImageMagick
3D Modeling/Rendering	Blender, FreeCAD, OpenSCAD
Video Editing	Shotcut, OpenShot, Kdenlive
Audio Processing	Audacity, Ardour
Office Suite	LibreOffice Writer/Calc/Impress
Game Development	Godot Engine
Notes/Knowledge Base	Obsidian, Zotero
Scientific Computing	Octave (MATLAB-compatible)
GIS Mapping	QGIS
Molecular Modeling	Avogadro, PyMOL
Blockchain	Ethereum staking tools

Project Advantages

Feature	CLI-Anything	Visual GUI Agents (UI-TARS)	Traditional RPA (UiPath)
Output Reliability	✅ Deterministic JSON	⚠️ Visual judgment, uncertainty	⚠️ Coordinate-dependent, fragile
Testability	✅ Complete test suite	❌ Hard to automate	⚠️ Limited
Speed	✅ Direct CLI invocation	⚠️ Screenshot-think-act loop	⚠️ Simulated operations are slow
API Dependency	None needed (wraps CLI)	None (visual control)	None (coordinate control)
Open Source	✅ Apache-2.0	✅	❌ Commercial
Reusability	✅ CLI-Hub ecosystem sharing	❌ Task-specific	⚠️ Workflow-specific

Detailed Analysis

1. SKILL.md: The Software Manual for AI Agents

This is the most forward-looking design in CLI-Anything. Traditional software only has documentation written for humans (README, man pages). SKILL.md is written specifically for AI agents:

# A complete SKILL.md example (Blender CLI)
---
name: blender-cli
version: 1.0.0
description: Blender 3D software CLI for AI agent automation
author: HKUDS/CLI-Anything
triggers:
  - "render 3d scene"
  - "create animation"
  - "blender"
  - "3d modeling"

commands:
  render:
    description: Render a Blender scene to image or animation
    usage: blender-cli render --scene <file> --output <path> [options]
    parameters:
      - name: --scene
        type: file_path
        required: true
        description: Path to .blend file
      - name: --output
        type: directory_path
        required: true
        description: Output directory for renders
      - name: --engine
        type: enum
        values: [CYCLES, EEVEE, WORKBENCH]
        default: EEVEE
    output_format: json
    example: |
      blender-cli render --scene scene.blend --output ./renders
      # Output: {"status": "success", "frames": 1, "output_path": "./renders/render0001.png"}

With this format, when an AI agent like Claude Code encounters a request to "render this 3D scene," it can load the relevant SKILL.md directly—knowing exactly which command to call, what parameters to pass, and what output format to expect. No manual configuration needed.

2. CLI-Hub: Building a Shareable CLI Ecosystem

CLI-Anything isn't just a generation tool—it includes a package management ecosystem:

Creator: Generates harness for GIMP → Publishes to CLI-Hub
User: cli-hub install gimp → One-command install → AI agent immediately has access

This follows the exact same logic as npm, pip, and brew:
  - Someone contributes a high-quality harness
  - The community can reuse it directly—no need for everyone to regenerate
  - Continuous improvement through versioned updates

This design transforms CLI-Anything from "a tool one person uses" into "a community-maintained standard library of AI-software interfaces."

3. The Relationship with HKUDS OpenHarness

These two projects from the same team form a natural technical stack:

OpenHarness (No. 96)
    ← Agent runtime infrastructure
    ← Tool-call engine, memory management, permission governance
    ← Provides "the agent's skeleton"

CLI-Anything (No. 104)
    ← Tool layer extension
    ← Wraps various software into OpenHarness-callable tools
    ← Provides "the agent's extended arms"

Used together:
  OpenHarness agent → CLI-Anything generated CLIs → Control Blender/GIMP/LibreOffice

Project Links & Resources

Official Resources

🌟 GitHub: https://github.com/HKUDS/CLI-Anything
📦 CLI-Hub Package Manager: pip install cli-anything-hub
🤝 Community: Feishu / WeChat developer communities (see GitHub README)
🐛 Software Wishlist: GitHub Issues wishlist template for requesting new app support

Target Audience

AI agent developers: Who need agents to control desktop software with no API
Automation engineers: Who want to programmatically drive graphical tools like GIMP and Blender
Open-source contributors: Who want to contribute new software harnesses to CLI-Hub, expanding the ecosystem
Enterprise automation teams: Who need to integrate legacy GUI applications into modern AI workflows

Summary

Key Takeaways

7-phase methodology: Analyze → Design → Implement → Plan Tests → Write Tests → Document → Publish—a complete automated CLI generation pipeline
SKILL.md: The software manual for AI agents—a standard format enabling "agents to automatically discover and use any tool"
80+ supported apps: GIMP, Blender, LibreOffice, Godot, FreeCAD—spanning creative, productivity, and engineering domains
CLI-Hub package management: Turns harnesses into shareable community assets instead of repeated work
HKUDS craftsmanship: 35.7k Stars, forming a complete technical stack with OpenHarness—"agent infrastructure + tool interfaces"

One-Line Review

CLI-Anything takes an extremely pragmatic but profoundly meaningful approach: rather than waiting for software vendors to design AI-friendly interfaces, it goes ahead and transforms the entire existing software ecosystem into a toolbox for AI agents.

Find more useful knowledge and interesting products on my Homepage

DEV Community