DEV Community: Priyank Malviya

7 CLI Tools Every Data Engineer Needs in 2025

Priyank Malviya — Sun, 26 Apr 2026 13:37:54 +0000

Why CLI Tools Still Matter

In a world of flashy GUIs and cloud consoles, CLI tools remain the secret weapon of productive data engineers.

Why?

Speed — No mouse hunting, no loading screens
Scriptability — Automate everything with bash, Python, or cron
Remote-friendly — SSH into any server and you're productive immediately
Composability — Pipe output between tools (tool1 | tool2 | tool3)
Lower resource usage — No Electron apps eating 2GB of RAM

The best data engineers I know live in the terminal. Here are the 7 CLI tools that will 10x your productivity in 2025.

1. Frosty — AI Agent for Multi-Database Operations 🥇

GitHub: https://github.com/Gyrus-Dev/frosty

Best for: Natural language database operations across Snowflake, PostgreSQL, SQL Server

Price: Free (open-source)

What It Does

Frosty is an AI agent that translates plain English into database operations. Instead of writing SQL or DDL manually, you type:

"show me my top 10 customers by revenue last quarter"
"set up MFA for all users without it"
"why is my warehouse spend up 40% this month?"
"create a data pipeline for S3 CSV ingestion with daily loads"

Frosty handles schema discovery, query generation, safety validation, and execution.

Why It's #1

Feature	Frosty	Traditional Tools
Natural language input	✅	❌
Multi-database support	✅ (Snowflake + Postgres + SQL Server)	Varies
Auto-generate DDL	✅	❌
Safety gates (block DROP)	✅	❌
Self-hosted	✅	Sometimes
Price	Free	$0–$$$

Real-World Use Cases

Data Engineering:

> "set up a data pipeline for S3 CSV ingestion"
# Generates: external stage, file format, table, COPY INTO, task schedule
# Time: 5 minutes vs. 2-3 hours manually

Security:

> "enforce MFA for all users without it"
# Generates ALTER statements, shows preview, executes with approval

Cost Governance:

> "why is warehouse spend up 40% this month?"
# Queries metering views, returns itemized breakdown with recommendations

Setup

git clone https://github.com/Gyrus-Dev/frosty.git
cd frosty
pip install -r requirements.txt
# Configure .env with credentials + API key
python -m src.frosty_ai.objagents.main

Model Support

OpenAI (GPT-4, GPT-4o)
Anthropic (Claude 3.5, Claude 3)
Google (Gemini 2.0, Gemini 1.5)
Ollama (local, free)

Swap models in one .env line — no code changes.

Safety

DROP unconditionally blocked in code
CREATE OR REPLACE requires explicit approval
Read-only modes for analyst roles
Audit logging of all executed statements

Verdict

Must-have for any data engineer working with Snowflake, PostgreSQL, or SQL Server. The time savings on repetitive tasks alone justify installation. Multi-database unified interface launching late April 2025.

Rating: ⭐⭐⭐⭐⭐ (5/5)

2. dbt (Data Build Tool) — SQL Transformations

Website: https://getdbt.com

Best for: Analytics engineering, SQL-based transformations in data warehouses

Price: Free (Core), $$$ (Cloud)

What It Does

dbt lets you write SQL transformations that are version-controlled, tested, and documented. You write SELECT statements; dbt handles DDL, dependencies, and orchestration.

Example Workflow

-- models/customers.sql
{{
  config(
    materialized='table',
    tags=['customers', 'core']
  )
}}

select
    customer_id,
    sum(revenue) as total_revenue,
    count(order_id) as order_count
from {{ ref('orders') }}
group by 1

Run with:

dbt run --select customers
dbt test --select customers
dbt docs generate

Key Features

Modular SQL — Use ref() to build dependency graphs
Testing — Built-in assertions (unique, not_null, accepted_values)
Documentation — Auto-generated data dictionary
CI/CD — Integrate with GitHub Actions, GitLab CI
Environments — Dev, staging, prod with separate schemas

When to Use It

✅ Building analytics models in Snowflake/BigQuery/Redshift

✅ Team collaboration on SQL transformations

✅ Need for testing and documentation

❌ Real-time streaming (use dbt + Airflow or dbt + Kafka)

❌ Non-SQL transformations (use Python + Pandas instead)

Verdict

Industry standard for analytics engineering. If you're doing SQL transformations in a data warehouse, dbt should be in your stack.

Rating: ⭐⭐⭐⭐⭐ (5/5)

3. pgcli / mycli — Enhanced Database Clients

GitHub: https://github.com/dbcli/pgcli (PostgreSQL)

GitHub: https://github.com/dbcli/mycli (MySQL/MariaDB)

Best for: Interactive SQL with auto-completion and syntax highlighting

Price: Free (open-source)

What They Do

These are drop-in replacements for psql and mysql with superpowers:

Auto-completion — Table names after FROM, column names after WHERE
Syntax highlighting — Colorized SQL in the terminal
Smart completion — Context-aware suggestions
Multi-line mode — Write complex queries across lines
Fuzzy search — Quickly find tables/columns

Example

# Instead of psql
psql -h localhost -U postgres -d mydb

# Use pgcli
pgcli -h localhost -U postgres -d mydb

Now type:

SELECT c.  # Auto-completes: customer_id, customer_name, created_at, ...
FROM customers c
WHERE c.   # Auto-completes column names again

Key Features

Feature	pgcli/mycli	psql/mysql
Auto-completion	✅	❌ (basic in psql)
Syntax highlighting	✅	❌
Smart completion	✅	❌
Multi-line mode	✅	Manual (`\e` in psql)
Fuzzy search	✅	❌

When to Use It

✅ Daily SQL work in PostgreSQL or MySQL

✅ Want better UX than default CLI

✅ Working with large schemas (auto-completion saves time)

❌ Need GUI features (ER diagrams, visual query builder)

❌ Working with unsupported databases (only Postgres/MySQL)

Verdict

Instant productivity boost for PostgreSQL and MySQL users. Installation takes 30 seconds; you'll wonder how you lived without it.

Rating: ⭐⭐⭐⭐⭐ (5/5)

4. ingestr — Data Copying Between Databases

GitHub: https://github.com/igorbarinov/ingestr

Best for: Copying data between databases without custom code

Price: Free (open-source)

What It Does

ingestr copies data from source to destination databases with support for incremental loads. No custom Python scripts needed.

Example

# Copy entire table
ingestr copy \
  --source postgres://user:pass@localhost/mydb \
  --source-table users \
  --destination snowflake://account/user/pass \
  --destination-table analytics.users

# Incremental load (append only)
ingestr copy \
  --source postgres://... \
  --source-table orders \
  --destination snowflake://... \
  --destination-table analytics.orders \
  --mode append \
  --incremental-key created_at

Supported Sources/Destinations

PostgreSQL
MySQL
SQL Server
BigQuery
Snowflake
Redshift
CSV/Parquet files

Key Features

Incremental loads — append, delete+insert, create+replace
Schema auto-detection — No manual DDL writing
Scheduling — Integrate with cron, Airflow, Prefect
Logging — Row counts, timing, error handling

When to Use It

✅ Copying data between databases regularly

✅ Need incremental loads (CDC-like behavior)

✅ Want to avoid custom ETL scripts

❌ Complex transformations (use dbt or Spark)

❌ Real-time streaming (use Kafka + connectors)

Verdict

Huge time-saver for data replication tasks. Replaces hours of custom script writing with a single command.

Rating: ⭐⭐⭐⭐ (4/5)

5. fzf — Fuzzy Finder

GitHub: https://github.com/junegunn/fzf

Best for: Quickly searching files, commands, history

Price: Free (open-source)

What It Does

fzf is a general-purpose fuzzy finder that lets you search anything in the terminal interactively.

Examples

# Search command history
# Press Ctrl+R, start typing, fzf finds matching commands

# Find and open a file
vim $(fzf)

# Search SQL files in a project
cat $(find . -name "*.sql" | fzf)

# Kill a process interactively
ps aux | fzf | awk '{print $2}' | xargs kill -9

# Preview file contents while searching
fzf --preview 'head -100 {}'

Key Features

Fuzzy matching — Type custmr and it finds customer_orders.sql
Interactive — Filter results as you type
Preview — See file contents before selecting
Keybindings — Ctrl+R for history, Ctrl+T for files
Integration — Works with vim, emacs, bash, zsh

When to Use It

✅ Large codebases with many files

✅ Frequent command history searches

✅ Want to navigate faster in the terminal

❌ Prefer GUI file explorers

❌ Working with small projects (less value)

Verdict

Universal productivity booster. Once you get used to fzf, you'll miss it in every other terminal session.

Rating: ⭐⭐⭐⭐⭐ (5/5)

6. jq — JSON Processor

GitHub: https://github.com/jqlang/jq

Best for: Parsing, filtering, transforming JSON in the terminal

Price: Free (open-source)

What It Does

jq is a lightweight command-line JSON processor. It's essential for working with APIs, logs, and JSON-heavy data pipelines.

Examples

# Pretty-print JSON
curl https://api.example.com/data | jq

# Extract specific fields
curl https://api.example.com/users | jq '.[] | {name, email}'

# Filter by condition
curl https://api.example.com/orders | jq '.[] | select(.status == "pending")'

# Aggregate data
curl https://api.example.com/orders | jq '[.[] | .amount] | add'

# Transform structure
curl https://api.example.com/data | jq '{total: .count, items: .results}'

Key Features

SQL-like syntax — Filter, map, reduce JSON
Chaining — Pipe multiple operations
Error handling — Graceful failures on malformed JSON
Colorized output — Readable by default

When to Use It

✅ Working with REST APIs

✅ Parsing JSON logs

✅ Transforming JSON for downstream systems

❌ XML data (use xmllint instead)

❌ Complex transformations (use Python + Pandas)

Verdict

Essential for any data engineer working with APIs or JSON data. The learning curve is steep but worth it.

Rating: ⭐⭐⭐⭐⭐ (5/5)

7. httpie — HTTP Client

Website: https://httpie.io

Best for: API testing and debugging

Price: Free (CLI), $$$ (Desktop app)

What It Does

httpie is a user-friendly HTTP client for the terminal. It's like curl but with readable syntax and colorized output.

Examples

# GET request
http GET https://api.example.com/users

# POST with JSON body
http POST https://api.example.com/users name="John" email="john@example.com"

# With authentication
http --auth user:pass GET https://api.example.com/protected

# Download a file
http --download GET https://example.com/file.zip

# Set custom headers
http GET https://api.example.com/data X-API-Key:abc123

Key Features

Feature	httpie	curl
Readable syntax	✅	❌ (flags everywhere)
Colorized output	✅	❌ (needs flags)
JSON auto-formatting	✅	❌ (pipe to jq)
Interactive mode	✅	❌
Download files	✅	✅

When to Use It

✅ Testing REST APIs

✅ Debugging webhook payloads

✅ Quick API experiments

❌ Production scripts (curl is more universal)

❌ Complex authentication flows (use Postman/Insomnia)

Verdict

Better than curl for interactive API work. The syntax is intuitive, and the output is immediately readable.

Rating: ⭐⭐⭐⭐ (4/5)

Honorable Mentions

These didn't make the top 7 but are still worth knowing:

Tool	Purpose
bat	`cat` with syntax highlighting and Git integration
ripgrep (rg)	Faster `grep` for code search
tmux	Terminal multiplexer for session management
httpie	Already covered, but worth repeating
peepDB	Quick database table inspection without SQL
Sling	Another data copying tool (alternative to ingestr)
Meltano	ELT orchestration with CLI interface

How to Get Started

This Weekend

Pick 2-3 tools and install them:

# Frosty (AI database agent)
git clone https://github.com/Gyrus-Dev/frosty.git
cd frosty && pip install -r requirements.txt

# pgcli (if you use PostgreSQL)
pip install pgcli

# fzf (fuzzy finder)
brew install fzf  # macOS
sudo apt install fzf  # Ubuntu

# jq (JSON processor)
brew install jq  # macOS
sudo apt install jq  # Ubuntu

# httpie (HTTP client)
brew install httpie  # macOS
sudo apt install httpie  # Ubuntu

Next Week

Try each tool on a real task (not just installation)
Replace one existing workflow with a CLI alternative
Share what you learned with your team

Long-Term

Build a personal toolkit of 10-15 go-to CLI tools
Script repetitive tasks using these tools
Contribute to open-source CLI projects you use

Final Thoughts

CLI tools aren't about rejecting modern GUIs — they're about having the right tool for the job.

Some tasks are better in a GUI (ER diagrams, visual query building). But for:

Quick data exploration
Repetitive admin tasks
Automation and scripting
Remote server work

CLI tools are unmatched.

Start with Frosty for database operations, add fzf for navigation, and jq for JSON work. You'll be shocked at how much faster you move.

Want to try Frosty?

🐙 GitHub — Star to get notified about multi-DB launch
💬 Discord — Join the community
📧 Contact — Questions or demo requests

About the author: This list was compiled by the team behind Frosty, an AI agent for multi-database operations. We live in the terminal and love sharing tools that make data engineering faster.

AI Agents for Database Management: Complete Guide 2025

Priyank Malviya — Sat, 25 Apr 2026 13:12:55 +0000

What Are AI Database Agents?

AI database agents are intelligent systems that understand natural language and translate it into database operations. Instead of writing SQL queries, crafting DDL statements, or memorizing CLI commands, you simply ask questions or give instructions in plain English.

Example:

Instead of:  SELECT customer_id, SUM(revenue) FROM orders 
             WHERE order_date >= '2025-01-01' 
             GROUP BY customer_id ORDER BY 2 DESC LIMIT 10;

You type:    "Show me my top 10 customers by revenue this year"

The agent handles schema discovery, query generation, execution, and returns results in a readable format often with explanations.

Key Use Cases for AI Database Agents

1. Natural Language Queries

Best for: Analysts, product managers, anyone who needs data but isn't SQL-fluent

"how many orders did we get last month?"
"what's the average order value by region?"
"compare revenue Q1 vs Q2 this year"

The agent discovers relevant tables, writes accurate SQL, and returns formatted results.

2. Automated Pipeline Creation

Best for: Data engineers tired of writing boilerplate DDL

"set up a data pipeline for S3 CSV ingestion with daily loads"

A single request can generate:

External stage configuration
File format definitions
Table schemas
COPY INTO statements
Task schedules for automation

What used to take hours happens in minutes.

3. Security Hardening

Best for: DBAs and security teams ensuring compliance

"enforce MFA for all users without it"
"apply password policies to meet SOC2 requirements"
"show me all users with excessive privileges"

Agents generate the ALTER statements, show you what will change, and execute with approval.

4. Cost Governance

Best for: FinOps teams managing cloud database spend

"why is my warehouse spend up 40% this month?"
"which queries are consuming the most credits?"
"recommend warehouse sizing for my workload"

Agents query metering views, identify anomalies, and suggest optimizations.

5. Multi-Database Operations

Best for: Teams managing Snowflake + PostgreSQL + SQL Server simultaneously

The next generation of agents (coming soon in tools like Frosty) will let you operate across multiple databases from a single interface — no more context-switching between tools.

AI Agents vs. Traditional Database Tools

Feature	AI Agents	DBeaver	DataGrip	SSMS / Snowsight
Natural Language Input	✅	❌	❌	❌
Auto-generate DDL from description	✅	❌	Partial	❌
Multi-step workflow automation	✅	❌	❌	❌
Safety gates (block DROP, require approval)	✅	❌	❌	❌
Schema-aware query generation	✅	❌	❌	❌
Cross-database operations	Emerging	✅	✅	❌
Price	Free–$$	Free–$$$	$25/mo	Free

Key insight: Traditional tools are excellent for manual SQL work. AI agents excel at automation, accessibility, and reducing repetitive tasks.

How to Choose an AI Database Agent

1. Safety Architecture

Ask: Can the agent execute destructive commands without my approval?

Look for:

Hard-coded blocks on DROP, TRUNCATE, DELETE FROM prod
Approval prompts for CREATE OR REPLACE
Read-only modes for analyst roles
Audit logging of all executed statements

2. Model Flexibility

Ask: Am I locked into one LLM provider?

Look for:

Support for OpenAI, Anthropic, Google, Ollama
Easy model switching via config (no code changes)
Local model support for cost-sensitive deployments

3. Database Coverage

Ask: Which databases does it support?

Current landscape:

Snowflake: Most mature AI agent support (Frosty, Snowflake Cortex, dbForge AI)
PostgreSQL: Growing support (pgEdge, Xata Agent, custom agents)
SQL Server: Microsoft-integrated tools (GitHub Copilot in SSMS, Azure OpenAI)
Multi-database: Emerging category (Frosty merging support in 2025)

4. Deployment Model

Ask: Where does the agent run?

Model	Pros	Cons
Self-hosted	Full control, credentials stay local, no SaaS fees	You manage infrastructure
SaaS	Zero setup, managed updates	Credentials leave your environment, recurring costs
Hybrid	Best of both	More complex setup

5. Community & Support

Ask: Is there an active user community?

Look for:

Active Discord/Slack community
Regular updates and changelog
Template gallery or recipe sharing
Responsive maintainers

Real-World Example: Frosty for Snowflake

Frosty is an open-source, self-hosted AI agent for Snowflake that demonstrates best practices in the category.

Key features:

153 specialist agents covering data engineering, security, governance, and admin
Two-layer safety system (prompt-level + hard-coded gates)
Supports OpenAI, Anthropic, Google, and local Ollama models
Self-hosted — credentials never leave your machine
Free and open-source

Example workflow:

# Clone and setup
git clone https://github.com/Gyrus-Dev/frosty.git
cd frosty
pip install -r requirements.txt

# Run and ask a question
python -m src.frosty_ai.objagents.main

> "set up MFA for all users without it"

Frosty inspects your users, generates ALTER statements, shows you what will change, and executes with approval.

What makes it different:

Safety-first architecture — DROP is unconditionally blocked in code
Context-aware — Inspects live environment before planning
Specialist agents — 153 focused agents vs. one generalist
No SaaS fees — You only pay for LLM tokens

Building Your Own AI Database Agent

If existing tools don't fit your needs, you can build a custom agent. Here's the minimal architecture:

Core Components

User Input (Natural Language)
         │
         ▼
Intent Classifier ──► Route to specialist agent
         │
         ▼
Schema Inspector ──► Discover tables, columns, relationships
         │
         ▼
SQL Generator (LLM) ──► Generate query/DDL
         │
         ▼
Safety Gate ──► Block/approve dangerous operations
         │
         ▼
Executor ──► Run against database
         │
         ▼
Result Formatter ──► Return to user

Key Libraries

Purpose	Library
LLM orchestration	LangChain, Google ADK, OpenAI SDK
Database connectivity	SQLAlchemy, Snowflake Connector, psycopg2
CLI interface	Rich, prompt_toolkit, Click
Safety validation	Custom regex + AST parsing

Safety Must-Haves

# Example safety gate (simplified)
def execute_query(query: str) -> dict:
    # Hard blocks
    blocked_patterns = ["DROP ", "TRUNCATE ", "DELETE FROM prod."]
    for pattern in blocked_patterns:
        if pattern.upper() in query.upper():
            return {"success": False, "message": f"Blocked: {pattern}"}

    # Approval required
    if "CREATE OR REPLACE" in query.upper():
        user_approval = input(f"Execute?\n{query}\n(yes/no): ")
        if user_approval.lower() != "yes":
            return {"success": False, "message": "User declined"}

    # Execute
    return snowflake_connection.execute(query)

The Future of AI Database Agents

What's Coming in 2025-2026

Multi-Database Unification
- Single interface for Snowflake + PostgreSQL + SQL Server + more
- Cross-database queries and pipelines
- Unified security and governance
Productivity Tool Integrations
- Agents for Confluent (Kafka), Asana, Notion
- Automated workflows between databases and business tools
- "Create a Jira ticket when this query returns X"
Vector Database Operations
- Natural language operations on vector embeddings
- RAG pipeline creation
- Semantic search across structured + unstructured data
Dynamic Visualizations
- "Show me a chart of revenue over time"
- Auto-generate Streamlit dashboards
- Interactive data exploration
Communication Integrations
- "Email this report to the team every Monday"
- "Send a Slack alert when warehouse spend exceeds $X"
- Automated reporting workflows

Getting Started Today

If You Use Snowflake

Try Frosty — It's free, open-source, and production-ready.

GitHub: https://github.com/Gyrus-Dev/frosty
Discord: https://discord.gg/GpgVzJ6ku
Setup time: ~10 minutes

If You Use PostgreSQL

Explore options:

pgEdge Agentic AI Toolkit
Xata Agent
Custom agents with LangChain + psycopg2

If You Use SQL Server

Explore options:

GitHub Copilot in SSMS
Azure OpenAI integration
dbForge AI Assistant

If You Use Multiple Databases

Watch this space. Multi-database AI agents are emerging in 2025. Frosty is merging PostgreSQL and SQL Server support into a unified interface — star their repo to get notified.

Final Thoughts

AI database agents aren't replacing database expertise — they're amplifying it. You still need to understand:

Your data model
Security requirements
Performance implications
Governance policies

What agents do is eliminate the repetitive work: writing boilerplate DDL, debugging syntax errors, switching between tools, and manually executing multi-step workflows.

The best time to start experimenting is now. Pick one agent, try it on a non-production database, and measure how much time it saves.

Want to see AI agents in action?

About the author: This guide was written by the team behind Frosty, an open-source AI agent for Snowflake. We're building the future of agentic data operations.

Frosty : 150 + AI Open Source Sub- Agents to Automate Snowflake

Priyank Malviya — Thu, 26 Mar 2026 14:18:09 +0000

With the advancement in AI, most teams are still not sure whether to use AI for their data or not. I feel the only way to empower them is to give them the control. That's why we released our AI Agent #Frosty as open source project on March 25th. Teams can own it and pay nothing beyond actual token usage.

What is Frosty?

Frosty is an AI Agent that can do everything that you can do on Snowflake. Yes EVERYTHING is a very broad scope, but that is exactly what Frosty can do. It has its own google search agent, which can read snowflake documentation and understand best practices and implement them for you.

It has a roster of 153 special agents, that can take care of engineering, administrator, security, governance , research tasks. It understands the entire context of your Snowflake by using its Inspector agent.

Agent Breakdown by Function

Engineer : 34 Agents
Administrator : 16 Agents
Security : 14 Agents
Governance : 8 Agents
Inspector : 56 Agents
Account Monitor : 25 Agents.

Checkout the repository here:

Gyrus-Dev / frosty

AI Agent for Snowflake

███████╗██████╗  ██████╗ ███████╗████████╗██╗   ██╗
██╔════╝██╔══██╗██╔═══██╗██╔════╝╚══██╔══╝╚██╗ ██╔╝
█████╗  ██████╔╝██║   ██║███████╗   ██║    ╚████╔╝
██╔══╝  ██╔══██╗██║   ██║╚════██║   ██║     ╚██╔╝
██║     ██║  ██║╚██████╔╝███████║   ██║      ██║
╚═╝     ╚═╝  ╚═╝ ╚═════╝ ╚══════╝   ╚═╝      ╚═╝
                    ╰─ by Gyrus Inc ─╯

An open-source, self-hosted agentic framework that turns plain English into Snowflake operations.

Quick Start · Features · Architecture · Setup · Safety · Contributing · Get in Touch

If you find Frosty useful, please consider giving it a ⭐ — it helps others discover the project!

What is Frosty?

Frosty is a 153-agent system built by Gyrus Inc that lets you manage your entire Snowflake environment in plain English — from querying data to administering security policies.

"who are my top 10 customers by revenue last quarter?"
  → Returns a Markdown table, powered by live SQL
"set up MFA for all users without it"
  → Generates and runs the ALTER statements, with your approval

"why is my

…

View on GitHub

What can Frosty do?

Frosty can build entire pipeline in minutes.
Frosty can create roles, grant privileges and plan the least access privilege policy.
Frosty can implement password policies, session policies, network rules, network policies to safeguard your Snowflake.
Frosty can implement masking policies, row access policies , plan tags to tag every object for better governance.
Frosty can access the metadata of every object on your Snowflake to understand your setup and plan things accordingly.
Frosty can generate synthetic data to help developers do comprehensive testing.
Frosty can do Data profiling and explain you the gaps in your data.

What Frosty can't do?

It can't drop any objects , as it has two level of safeguards, one through prompt, the other through code. So even if for some reason Frosty decides to run a DROP query, the code blocks it out before it reaches Snowflake.
It can't run CREATE OR REPLACE statements, without approval.
It can't create dashboards, but being open source users can extend its capabilities.

Frosty is open source and free to use. We would love feedback, feature requests , contributions from Dev.to community.