DEV Community: SHREYA

The Great AI Comparison: Which Tool Fits Your Workflow Best? by Runner H

SHREYA — Thu, 26 Jun 2025 15:39:44 +0000

This is a submission for the Runner H "AI Agent Prompting" Challenge

What I Built

I built an automated AI Tool Comparison Generator using Runner H’s agent capabilities. This agent takes 4–5 popular AI tools (like ChatGPT, Gemini, Notion AI, Claude, and Perplexity), visits their websites, YouTube demo videos, and pricing pages, and auto-generates a detailed comparison table + infographic.

The goal is to help users, developers, and decision-makers quickly compare tools across critical features like output quality, speed, privacy, integrations, and pricing — without doing manual research.

What sets this apart? I connected the Runner H agent to Google Docs and Notion, enabling it to surf embedded documents, product changelogs, and deep-dive feature breakdowns that are often missed when scraping just public-facing websites.

Demo

Prompt

Compare the following 5 AI tools:

- Notion AI
- Google Gemini (Workspace integration)
- ChatGPT (OpenAI)
- Claude (Anthropic)
- Perplexity AI

Visit each tool’s:

- Official website
- YouTube demos and walkthroughs
- Pricing pages and feature docs

Create a comparison table and a visually engaging infographic based on these 10 features:

- Core Capabilities – What can it do (writing, coding, research, summarization, etc.)?
- User Interface & Experience – Ease of use, visual design, onboarding
- Integration Support – Works with Google Docs, Slack, APIs, browser extensions, etc.
- Output Quality – Accuracy, nuance, style, tone control
- Speed & Performance – Response time, downtime reports, performance under load
- Customization Options – Custom instructions, personas, plugins
- Collaboration Tools – Shared chats, team spaces, export options
- Pricing & Plan Tiers – Free tier vs pro plans, monthly cost, limits
- Privacy & Data Usage – Data retention, user control, enterprise options
- Unique Selling Points – Any special features, modes, or tech differentiators
End with a 1-line summary for each tool:

“Best for ____” (e.g., students, researchers, product managers, etc.)

✅ Below attached the Demo link from Runner H

AI Comparison Tool

✅ Attached the Youtube Video

Screenshots

Connecting to Google Docs & Notion In this first step, the Runner H agent is configured to access and surf embedded Google Docs and Notion pages. These sources often contain detailed product specs, roadmaps, or changelogs that aren't available on public landing pages — making this step essential for capturing rich, contextual insights about each AI tool.

Searching Information for 5 AI Tools The agent then begins its information-gathering phase, visiting the official websites, YouTube demos, pricing pages, and documentation for the selected AI tools — ChatGPT, Google Gemini, Claude, Notion AI, and Perplexity AI. This ensures it collects up-to-date and accurate data across 10 comparison features.

Feature-by-Feature Comparison Table

Once the data is extracted, Runner H compiles it into a structured comparison table, highlighting differences across 10 key features like core capabilities, output quality, speed, integrations, customization, and pricing. This table is automatically exported to Google Docs for easy viewing, editing, or sharing.

Summary & Recommendations

Finally, the tool generates a summary row for each AI assistant, offering a concise “Best for ___” recommendation. This helps users quickly identify which tool is ideal for students, creators, teams, technical users, or enterprise adoption — turning hours of manual comparison into a one-minute insight.

Generated PDF
Link

How I Used Runner H

I used Runner H’s agent framework to:

Input Prompt Structure: Created a structured prompt that asks the agent to compare 5 specific AI tools across 10 detailed features.
Live Web Scraping: The agent navigates to the official sites, YouTube demos, and pricing pages of each tool.
Data Aggregation: Runner H extracts key points from various sources and compiles them into a table.
Infographic Generation: The agent outputs a ready-to-publish infographic summarizing findings visually.
Export: The results are exported directly into a Google Docs and image asset, ideal for content creators or analysts.

Use Case & Impact

✅ Product Managers comparing LLM-powered platforms
Quickly evaluate AI tools based on technical depth, integration support, and team collaboration features. Helps make data-driven decisions aligned with product goals and timelines.
✅ Founders & CTOs evaluating which AI to embed
Gain clarity on performance, API access, privacy policies, and scalability. Reduces risk when selecting the best-fit AI solution for long-term product infrastructure.
✅ Content Creators building blogs, reels, or carousels
Instantly generate comparison tables and infographics with accurate, current data. Saves hours of research and design, letting creators focus on storytelling and growth.
✅ Educators and Analysts creating lecture slides or decks
Pull real-world insights from official docs and demos to create impactful learning content. Keeps academic and professional materials fresh, relevant, and engaging.

🚀 Why It Matters
By combining Runner H with deep-link surfing into Google Docs and Notion, this tool uncovers highly contextual insights often missed by surface scrapers. Whether you're benchmarking tools, preparing content, or comparing platforms for adoption — this agent saves hours of manual work and produces sharable, up-to-date outputs in minutes.

Social Love

I’m thrilled to share my submission for the Runner H with the community! 🚀
This project represents countless hours of creativity, coding, and collaboration. I’m proud of what I’ve built and excited to see how others push boundaries too!

🔗 Check it out and let me know what you think!
// Detect dark theme var iframe = document.getElementById('tweet-1938262441055379893-954'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1938262441055379893&theme=dark" }

Made with ❤️
Huge thanks to Runner H and Dev community for hosting this awesome challenge! 🙌

Refactor Advisor and Analysis Automation Tool- Smart Code Amazon Q CLI Tool

SHREYA — Sun, 11 May 2025 11:51:10 +0000

This is a submission for the Amazon Q Developer "Quack The Code" Challenge: Crushing the Command Line

What I Built

I built RefactorAdvisor because I often found myself overwhelmed revisiting old Python projects 🧠💻 — the kind where functions stretch hundreds of lines, imports pile up, and clarity fades away. Like many developers, I wanted to clean things up but didn’t always have the time (or patience) for deep manual reviews. Traditional linters felt too shallow, and I wanted something smarter. That’s when I discovered Amazon Q 🤖 — a tool that could understand context and suggest meaningful improvements. So, I created RefactorAdvisor: a CLI tool that walks through your project 📁, splits long files into manageable chunks ✂️, sends them to Amazon Q for analysis, and returns actionable refactoring tips 🛠️. Whether you just want quick insights or a full Markdown report 📄, it helps bring order back to your code — one chunk at a time.

Key Features of RefactorAdvisor

🗂️Smart Project Traversal 🔍 Recursively scans a specified project directory

📄 Identifies all .py source files for analysis

$ python refactor_advisor.py --path ./my_project

✂️ Chunking Large Files 🧩 Breaks large files into ~100-line chunks (adjustable via CLI)

🔄 Keeps analysis consistent with Amazon Q’s input limits

$ python refactor_advisor.py --path ./app --chunk-size 80

🤖 Amazon Q Integration 📤 Sends each chunk to Amazon Q CLI via subprocess

🧠 Asks Q to:

Summarize the chunk

Identify code smells

Suggest refactorings based on:

DRY
SOLID
Readability
Performance

⚠️ Requires AWS CLI & Amazon Q setup
Uses aws q or amazon-q under the hood

📊 Organized Output 🎨 Displays feedback in the terminal with:

✅ Sectioned summaries

🎯 Color-coded highlights

📝 Optionally writes reports in Markdown per file:

Example: analysis/utils_refactor_report.md

$ python refactor_advisor.py --path ./api --output ./refactor_reports

Customizable CLI 🧪 Simple and flexible interface using argparse

$ python refactor_advisor.py \
    --path ./project_dir \
    --output ./analysis_reports \
    --chunk-size 120

Demo

Screenshots and Report Files
💻 Chunking the Python file with Amazon Q 🐍

💻 Chunking, Give Code and Function Summary, Code Fixes and Refactoring Suggestions🐍

- Below Attached the Brief reports on the file which was analyzed in the below drive in .txt and .md format generated by the tool

Brief Analysis Report

Code Snippets
🧩 Part 1: Imports and Setup

import os
import argparse
import subprocess
from pathlib import Path

📌 Explanation:

Imports necessary modules:

os, Path for filesystem operations
argparse for CLI argument parsing
subprocess for interacting with Amazon Q via terminal

🧩 Part 2: Find Python Files

def find_python_files(directory):
    """Recursively finds all .py files in the given directory."""
    py_files = []
    for root, _, files in os.walk(directory):
        for file in files:
            if file.endswith('.py'):
                py_files.append(os.path.join(root, file))
    return py_files

📌 Explanation:

Walks through the given directory and subfolders
Collects paths of all .py files for later analysis

🧩 Part 3: Chunking Source Code


def chunk_file(file_path, chunk_size=100):
    """Splits a Python file into chunks of N lines."""
    with open(file_path, 'r', encoding='utf-8') as f:
        lines = f.readlines()

    chunks = []
    for i in range(0, len(lines), chunk_size):
        chunk = lines[i:i + chunk_size]
        chunks.append((i // chunk_size + 1, ''.join(chunk)))
    return chunks

📌 Explanation:

Reads the content of a file
Divides it into chunks of chunk_size lines (default is 100)
Returns a list of tuples: (chunk_number, code_text)

🧩 Part 4: Analyze Chunk with Amazon Q

def analyze_chunk(chunk_code):
    """Sends a chunk of code to Amazon Q CLI for analysis."""
    process = subprocess.Popen(
        ['amazon-q', 'analyze'],
        stdin=subprocess.PIPE,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        text=True
    )
    stdout, stderr = process.communicate(chunk_code)
    return stdout if stdout else stderr

📌 Explanation:

Uses subprocess.Popen() to call the Amazon Q CLI
Sends a chunk of code for analysis
Captures and returns either the standard output or error

🧩 Part 5: Write Results to Files

def write_report(file_name, results, output_dir):
    """Writes the analysis results to Markdown and TXT files."""
    output_path_md = Path(output_dir) / f"{Path(file_name).stem}_refactor_report.md"
    output_path_txt = Path(output_dir) / f"{Path(file_name).stem}_refactor_report.txt"

    with open(output_path_md, 'w', encoding='utf-8') as f_md, open(output_path_txt, 'w', encoding='utf-8') as f_txt:
        for result in results:
            f_md.write(result + '\n\n')
            f_txt.write(result + '\n\n')

📌 Explanation:

Writes the combined results of all chunks from one file into:
.md for rich text reports
.txt for plain text output
Uses the file name as the base for the report name

Code Repository

🔍 Explore the complete source code on GitHub:
💻 Built with Python 🐍 and designed to integrate smoothly with Amazon Q 🤖 for smarter code reviews and refactoring advice!

Refactor Advisor & Analysis Automation Tool

How I Used Amazon Q Developer

Scenario:
You are tasked with refactoring a large Python codebase that spans multiple files and has accumulated technical debt over time. The code is hard to maintain, and improving its quality manually seems time-consuming. To automate the process and ensure you don't miss any potential improvements, you decide to use the Refactor Advisor, a tool powered by Amazon Q Developer.

Solution: Refactor Advisor with Amazon Q Developer CLI
Tool Setup and Initialization 🛠️

The Refactor Advisor Python script (refactor_advisor.py) is designed to automate the process of analyzing and refactoring Python code.

The script uses Amazon Q Developer CLI to send code chunks to Amazon Q for analysis and improvement suggestions.

The tool is set up with the following command-line arguments:

--path: Path to the project directory you want to analyze.

--output: (Optional) Path to store the output Markdown report.

--chunk-size: Defines the number of lines in each chunk for Amazon Q to process (default is 100).

Project Traversal and File Detection 🔍

You run the following command to start the analysis:

python refactor_advisor.py --path ./my_project --output ./analysis The tool recursively traverses the ./my_project directory to find all Python files (.py).
Result: The tool identifies 30 Python files in the project directory that need analysis.

Breaking Files into Chunks 🔨

Each Python file is read and broken into smaller chunks of approximately 100 lines (the chunk size is adjustable).
Large functions or classes may be split across multiple chunks to ensure each part is manageable and fits Amazon Q's input size.
Sending Chunks to Amazon Q for Analysis 📤
For each chunk, the script sends a request to Amazon Q Developer CLI with a prompt asking for:
A summary of the code: A high-level overview of what the chunk of code is doing.
Detected code smells or anti-patterns: Issues like duplicated code, overly complex functions, long methods, or violation of best practices.
Refactoring suggestions: Based on DRY (Don’t Repeat Yourself), SOLID principles, readability, and performance, Amazon Q suggests how to improve the code.
Each chunk is processed individually to ensure detailed feedback for each section of the code.

Receiving Feedback from Amazon Q 🔄

Amazon Q returns structured feedback in response to each chunk, which includes:

A brief summary of the code.
Code smells detected in the chunk (e.g., code duplication, long methods).
Suggestions for improvement, including:
Refactoring functions to be smaller.
Improving variable names for clarity.
Combining duplicate code to follow the DRY principle.
Breaking down large classes or functions to adhere to SOLID principles.
Displaying the Results 🖥️
The tool collects all the feedback from Amazon Q and displays it in the terminal.
Color-coding and sectioning are used to distinguish between:
Green: Suggested improvements.
Yellow: Warnings (code smells detected).
Blue: Code summaries.
The results are presented in a clear and structured way, making it easy for developers to understand and take action.
Optional Markdown Reports 📄 The results are presented in a clear and structured way, making it easy for developers to understand and take action.

🔍 Learnings & Challenges Faced

✅ Learnings

Integrated Amazon Q Developer CLI with Python using subprocess.
Built a flexible CLI tool using argparse.
Implemented file chunking logic for large Python files.
Parsed and formatted Q’s output for terminal and markdown.
Learned prompt engineering for code summarization and refactoring advice .

⚠️ Challenges

Handling multiline and unstructured Q responses.
Managing subprocess timeouts and rate limits.
Ensuring accurate chunk splits without breaking logic.
Dealing with file encoding errors.
Validating Amazon Q CLI installation and user configuration.

Challenges & ✅ Solutions

- Unstructured Q Responses

✅ Solution: Used textwrap, regex, and custom parsing logic to extract summaries, issues, and suggestions reliably from mixed Q outputs.

- Subprocess Timeouts / Rate Limits

✅ Solution: Added try/except around subprocess.run() with timeout parameters. Implemented delays between chunk requests if needed using time.sleep().

try:
    result = subprocess.run(cmd, capture_output=True, text=True, timeout=30)
except subprocess.TimeoutExpired:
    print("⚠️ Amazon Q CLI timed out. Retrying`...")

- Chunking Logic Accuracy

✅ Solution: Used a line counter and buffer to break files logically without splitting mid-function or mid-class (where possible). Configurable chunk size ensures flexibility.

for i in range(0, len(lines), chunk_size):
    chunk = lines[i:i + chunk_size]

- File Encoding Issues

✅ Solution: Opened files using encoding='utf-8' and handled UnicodeDecodeError gracefully.

   with open(file_path, 'r', encoding='utf-8') as f:
        content = f.read()
except UnicodeDecodeError:
    print(f"⚠️ Skipping {file_path} due to encoding error.")
Q CLI Configuration Validation

✅ Solution: Checked if q CLI is installed using shutil.which() and printed a friendly error if not found.


if shutil.which("q") is None:
    print("❌ Amazon Q CLI not found. Please install and configure it first.")
    sys.exit(1)

Conclusion
This tool revealed common issues such as lack of classes, long functions, inconsistent naming, and missing documentation. Many chunks contain code smells like hardcoded values, redundant logic, and poor error handling.

To improve the codebase, refactor large functions, adopt consistent naming, improve documentation, and apply object-oriented design where possible. Addressing these areas will enhance maintainability, readability, and reduce technical debt.

Reachout : Gmail : Shreyan11111@gmail.com