DEV Community: Chanyeong Yun

Beyond the Hype: What I Learned from a 45-Minute AI Research Agent Lab

Chanyeong Yun — Sat, 01 Aug 2026 02:50:53 +0000

AI Agents are everywhere right now. We've all seen the flashy demos and read countless articles, but I wanted to understand how a research agent actually works beyond the marketing buzz.

So I spent about 45 minutes completing Microsoft's Applied Skills – Generate Reports with AI Research Agents.

Unlike a typical certification exam, this is a hands-on lab that runs inside a virtual machine. Instead of answering multiple-choice questions, you're guided through building an AI-assisted reporting workflow using Microsoft 365 Copilot Chat and Research Agents.

💡 What Stood Out to Me

The most interesting part wasn't the AI model itself—it was the workflow.

Rather than asking a single prompt and hoping for a good answer, the Research Agent follows a structured process:

Analyzes uploaded documents
Extracts relevant information
Synthesizes the findings
Produces a structured report

Watching that process happen step by step made the concept of an "AI Agent" much more tangible than simply reading about it in a tech blog.

🔍 Is It Worth Trying?

If you're curious about AI Agents or Microsoft 365 Copilot, I'd say yes.

Quick Setup: The lab is short (around 45 minutes) and runs entirely in a pre-configured VM, so there's zero local setup required.
Practical Insights: It provides a realistic introduction to how AI-assisted document automation works in an enterprise workflow.
Bonus Credential: You earn an official Microsoft Applied Skills badge upon completion—which is a nice bonus, but honestly not the main reason I'd recommend it.

If you've been wanting to experiment with AI Agents firsthand without committing to a massive course, this interactive lab is a great place to start.

👉 Try the hands-on lab on Microsoft Learn

Have you tried any hands-on AI Agent labs recently? I’d love to hear which platforms or tools you’ve found most useful!

PDF Parsing Is the Hidden Bottleneck in Your RAG Pipeline

Chanyeong Yun — Mon, 27 Jul 2026 09:57:20 +0000

When a RAG system returns hallucinatory or plain wrong answers, our first instinct is usually to blame the retriever or swap the LLM for a bigger one.

However, while stress-testing an end-to-end RAG pipeline on table-heavy Korean documents, I realized the real culprit was located one stage earlier: the PDF parser.

If your parser shreds a table into a stream of flat text, no amount of prompt engineering or high-dimensional embeddings can recover the lost structural context.

How Bad Parsing Destroys Retrieval

Context Pollution: When basic parsers inject broken encoding, missing whitespace, or garbage metadata into text chunks, embedding models end up vectorizing noise. Retrieval precision drops immediately.
Structure Collapse: This is the ultimate killer for table-heavy PDFs. Flattening a multi-column table destroys the explicit relationships between headers, cells, and values. The LLM then hallucinates connections between unrelated numbers.

In my test cases, every failure regarding numeric thresholds traced directly back to a table that had been flattened at ingestion.

Benchmarking Local PDF Parsers (Korean + Tables)

I benchmarked standard open-source tools against complex Korean PDF documents containing merged cells and multi-line headers:

Parser / Library	Table Structure Preservation	Korean Encoding	Security / Local	Notes
PyPDFLoader (LangChain)	⭐	❌ Frequent corruption	✅ Fully local	Fast, but treats table text as a raw line-by-line dump. Structure is completely lost.
pdfplumber	⭐⭐⭐⭐	✅ Clean	✅ Fully local	Pure Python. Native `extract_tables()`, allows custom spatial layout & coordinate-based extraction.
LlamaParse / Commercial APIs	⭐⭐⭐⭐⭐	✅ Clean	❌ External API	Incredible extraction accuracy, but sending sensitive documents to third-party APIs was a non-starter for security.

Why I Built a `pdfplumber` Pipeline

Structure-Aware: It preserves row/column coordinate boundaries, allowing us to reconstruct markdown tables accurately.
Local & Lightweight: No external API dependency, zero risk of data exposure.

Overcoming `pdfplumber` Limitations with Post-Processing

While pdfplumber is far better than raw text dumps, it isn't magic out of the box. I had to implement a few custom post-processing rules to make the output LLM-ready:

import pdfplumber

def extract_and_clean_tables(pdf_path):
    cleaned_tables = []
    with pdfplumber.open(pdf_path) as pdf:
        for page in pdf.pages:
            tables = page.extract_tables()
            for table in tables:
                # 1. Handle merged cells returning as None
                # 2. Normalize in-cell line breaks (\n -> " ")
                cleaned_table = []
                for row in table:
                    cleaned_row = [
                        (cell.replace("\n", " ") if cell else "")
                        for cell in row
                    ]
                    cleaned_table.append(cleaned_row)
                cleaned_tables.append(cleaned_table)
    return cleaned_tables

Merged Cell Handling: Blank-fill None values left behind by merged spans.
In-cell Newlines: Replace internal \n with spaces so multi-line text cells don't fake a new row boundary.

Results: Golden Set Evaluation

I re-ran a test set of domain-specific queries against both pipelines:

Query Type	Default PyPDFLoader Pipeline	Custom pdfplumber Pipeline
Numeric Thresholds (Reference tables)	❌ Failed (retrieved wrong row)	✅ Exact Match
Formula / Term Definitions	❌ Failed (text mashed together)	✅ Exact Match
Recurring Schedule Values	⚠️ Partial / Hallucinated	✅ Exact Match

Key Takeaways

Parsing quality IS retrieval quality: Don't waste weeks tuning chunk sizes or fine-tuning embeddings if your input text is already corrupted at parsing time.
Minimize context pollution: Preserve table boundaries (e.g., convert to clean Markdown tables before chunking) to give the retriever and LLM the best chance of success.

Next Steps

Exploring local open-source Vision Language Models (e.g., Qwen2-VL or Florence-2) to parse extremely complex, nested image-based tables without sending data off-premise.

31.8x Speedup by Changing One File: Async Embedding Calls on AWS Bedrock

Chanyeong Yun — Mon, 27 Jul 2026 07:16:14 +0000

I was testing a RAG ingestion pipeline and noticed something painful: ingesting a single document took nearly 50 seconds.

Checking the metrics revealed that the CPU was doing almost nothing. The app wasn't compute-bound — it was simply sitting idle, waiting for sequential HTTP network round-trips to return.

By converting the blocking embedding module into an asynchronous workflow, processing time dropped from 49.61 seconds to 1.56 seconds with zero infrastructure changes.

Benchmark Setup

Model: Amazon Titan Text Embeddings V2 (AWS Bedrock)
Dataset: 33 text chunks from a single document
Region: us-east-1
Test: Sequential blocking requests vs. concurrent asynchronous requests

Results

Approach	Time	Difference
Sequential (`requests` blocking loop)	49.61s	Baseline
Concurrent (`aiohttp` + `asyncio.gather`)	1.56s	31.8× faster

Why the Gap Was So Massive

Sequential (Blocking I/O)

Each request must complete before the next one starts.

chunk 1 ──> request ──> ⏳ wait (~1.5s) ──> response
chunk 2 ──> request ──> ⏳ wait (~1.5s) ──> response
chunk 3 ──> request ──> ⏳ wait (~1.5s) ──> response
...
Total Time ≈ (33 chunks × ~1.5s) ≈ 49.6s

Since network latency dominates the runtime, the event loop sits completely unused.

Concurrent (Non-blocking Async)

Requests are dispatched concurrently; total time collapses down to the duration of the slowest single request.

chunk 1  ──> request ──────────────> response
chunk 2  ──> request ──────────────> response
chunk 3  ──> request ──────────────> response
...        (in flight concurrently)
Total Time ≈ slowest single request ≈ 1.56s

The Code Change

Before: Sequential Execution

import requests

def generate_embeddings(texts: list[str]) -> list[list[float]]:
    all_embeddings = []
    for text in texts:
        response = requests.post(url, headers=headers, json={"inputText": text})
        vector = response.json().get("embedding", [])
        all_embeddings.append(vector)
    return all_embeddings

After: Concurrent Execution

import asyncio
import aiohttp

async def fetch_embedding(session: aiohttp.ClientSession, text: str) -> list[float]:
    async with session.post(url, headers=headers, json={"inputText": text}) as res:
        data = await res.json()
        return data.get("embedding", [])

async def generate_embeddings_async(texts: list[str]) -> list[list[float]]:
    async with aiohttp.ClientSession() as session:
        tasks = [fetch_embedding(session, text) for text in texts]
        return await asyncio.gather(*tasks)

Note: If you're using boto3 instead of raw HTTP requests, look at aioboto3, or offload the blocking SDK calls with asyncio.to_thread() to get similar non-blocking behaviour.

How It Scales

Chunks	Sequential (est.)	Concurrent (est.)
31	49.61s	1.56s
100	~160s (2.7 min)	~2–3s
500	~800s (13 min)	~3–5s
1,000	~1,600s (26 min)	~5–10s

The more chunks you ingest, the more the concurrent approach pays off — batch ingestion jobs that took half an hour now finish in seconds.

Key Takeaways

Check for idle waiting first. Before scaling up infrastructure or adding complex worker queues, verify whether your pipeline is simply blocked on I/O.
Mind the service quotas. Bedrock has per-region request-rate limits — as chunk counts grow into the thousands, cap concurrency (e.g. asyncio.Semaphore) before you hit them.
Low effort, high impact. Changing just one file (embedder.py) removed the embedding stage as a bottleneck in the pipeline.

DEV Community: Chanyeong Yun

Beyond the Hype: What I Learned from a 45-Minute AI Research Agent Lab

💡 What Stood Out to Me

🔍 Is It Worth Trying?

PDF Parsing Is the Hidden Bottleneck in Your RAG Pipeline

How Bad Parsing Destroys Retrieval

Benchmarking Local PDF Parsers (Korean + Tables)

Why I Built a pdfplumber Pipeline

Overcoming pdfplumber Limitations with Post-Processing

Results: Golden Set Evaluation

Key Takeaways

Next Steps

31.8x Speedup by Changing One File: Async Embedding Calls on AWS Bedrock

Benchmark Setup

Results

Why the Gap Was So Massive

Sequential (Blocking I/O)

Concurrent (Non-blocking Async)

The Code Change

Before: Sequential Execution

After: Concurrent Execution

How It Scales

Key Takeaways

Why I Built a `pdfplumber` Pipeline

Overcoming `pdfplumber` Limitations with Post-Processing