DEV Community: Nexus Labs

Taming LLMs: The Fight for Valid JSON in Our Webhooks

Nexus Labs — Wed, 04 Nov 2026 15:01:54 +0000

Alright, fellow builders. You know the drill: shiny new idea, integrate an LLM, everything's gonna be super smart and dynamic. That was me last week, sketching out a new webhook handler for our internal tools. The goal was simple, or so I thought: feed some unstructured text (think user feedback, support tickets, internal notes) to an LLM, and get back a perfectly structured JSON object. This JSON would then drive subsequent actions – like creating a task, updating a record, or flagging an issue.

My initial mental model was this:

python

Imagine some LLM API call

llm_output = llm_client.generate(prompt=user_text, schema=expected_json_schema)
parsed_data = json.loads(llm_output)

... off to Pydantic validation and processing

Cute, right? Naive, more like. The reality hit faster than a compiler error after a typo. My LLM, bless its token-generating heart, was consistently spitting out JSON that was almost there, but not quite. Sometimes it was missing a comma, sometimes an extra newline, other times it wrapped the JSON in markdown fences ( ), or worse, added conversational filler like "Here's your data:".

This wasn't just a nuisance; it was a showstopper. If json.loads() failed, the whole webhook just… stopped. No task created, no record updated. It took a full day of hair-pulling and muttering to myself before I started seeing the light.

What I Tried (and What Didn't Quite Stick)

Aggressive Prompt Engineering: My first instinct was to just yell at the LLM harder. "RETURN ONLY THE JSON. NO OTHER TEXT. DO NOT EXPLAIN." I even included the full Pydantic model as a string in the prompt. This helped marginally. The LLM would usually return JSON, but the edge cases still tripped it up. An empty string somewhere, a missed closing brace. Annoying.
Basic String Cleanup: I tried some rudimentary regex to strip markdown fences ( ) or leading/trailing whitespace. This felt super hacky. It worked for some cases, but I knew it was a ticking time bomb. One slightly different output format from the LLM, and my regex would break or, worse, mangle valid JSON.
try-except with json.loads: This is, of course, essential. You must wrap your JSON parsing. But simply catching the JSONDecodeError just tells you it failed, not why or how to fix it. My webhook would just log the error and bail, which wasn't good enough for an automated system.
Pydantic Validation (after json.loads): Once I did get valid JSON, Pydantic v2 was awesome for schema validation. It ensured the types were correct, required fields were present, etc. But again, if json.loads failed, Pydantic never even got a chance.

I was stuck in a loop: LLM gives bad JSON -> json.loads fails -> I'm sad. I needed something more robust than just hoping the LLM would be perfect every single time.

The Combo That Finally Fixed It (Mostly)

After a lot of trial and error, here’s the stack of strategies that finally brought some sanity to my webhook handler:

1. The LLM's Own JSON Mode (If Available)

This was a huge win. For LLMs like OpenAI's gpt-4-turbo or gpt-3.5-turbo, there's a specific response_format parameter you can pass. Setting response_format={'type': 'json_object'} is a game-changer. It instructs the model to only return valid JSON. This significantly reduced the malformed output right out of the gate.

python

Example using OpenAI Python client (simplified)

from openai import OpenAI
import json

client = OpenAI()

def get_structured_output(prompt_text, system_message):
response = client.chat.completions.create(
model="gpt-4-turbo-preview",
messages=[
{"role": "system", "content": system_message},
{"role": "user", "content": prompt_text}
],
response_format={"type": "json_object"} # <<< The magic!
)
return response.choices[0].message.content

... then parse with json.loads and validate with Pydantic

Even with this, it's not 100% foolproof, especially with really complex or ambiguous prompts, but it gets you 95% of the way there.

2. A Resilient Retry Loop with Feedback

This was the real key to robustness. Instead of just failing on bad JSON, I implemented a retry mechanism that fed the error back to the LLM.

Here’s the basic flow:

Attempt 1: Send the prompt to the LLM (with json_object format if possible).
Parse and Validate: Try json.loads(). If it works, then pass the dict to your Pydantic model for validation. If either fails, catch the specific error.
Retry (with context): If parsing/validation fails, create a new prompt for the LLM. This prompt includes:
- The original user input.
- The malformed JSON the LLM previously produced.
- The exact error message from json.loads or Pydantic.
- A clear instruction: "The previous output was invalid. Here's the error: [...]. Please correct it."
Limit Retries: I set a strict limit, usually 2 or 3 retries. If it fails after that, then I log it as a legitimate failure and probably escalate it for human review.

This turns a hard failure into a soft loop, giving the LLM a chance to self-correct. It sounds simple, but it was surprisingly effective. The LLM, given its own error message, often nails it on the second try. My webhook went from failing on 10-15% of inputs to practically zero.

Wrapping Up

It turns out reliably getting structured data from an LLM isn't just about crafting the perfect initial prompt. It's about designing a system that acknowledges the LLM's inherent occasional weirdness and builds in mechanisms to recover. The combination of leveraging specific LLM features like json_object responses and implementing a smart retry loop with error feedback has been a lifesaver. My webhooks are happy, and so am I. Hopefully, this saves you some grey hairs on your next LLM project! 🚀

Taming My GitHub Backlog with a Local LLM: A Triage Game Changer

Nexus Labs — Tue, 03 Nov 2026 15:01:56 +0000

My sprint started like usual: staring down a mountain of GitHub issues. Some of them, bless their hearts, had comment threads longer than a novella. You know the ones – the original bug report, 10 comments debating a workaround, 5 more about a related edge case, and then buried somewhere, the actual resolution or next step. Triage felt less like managing work and more like an archaeological dig.

In the past, I'd just skim, open every link, try to piece together the narrative. It was agonizing, easily eating an hour or more just to get a handle on a handful of complex issues. This sprint, I was determined to do better. My first thought was a full-blown AI integration, but I'm an indie dev, not a bank. Paid LLM APIs felt like overkill for internal messy comments, plus I'm a bit wary of sending all my internal discussions to a third party. I needed something private, cheap, and effective.

That's when I remembered Ollama. I've been dabbling with local models, mostly for fun. Could I really throw a chunky GitHub thread at llama3 and get something useful back? Turns out, yes, but not without some nudging.

What Actually Fixed It

1. Getting the Data (Without Over-Engineering)

For my initial test, I just copy-pasted the entire comment history for a particularly gnarly issue into a text file. For real use, I quickly whipped up a little helper function using the gh CLI. It's simple, but it works:

python
import subprocess
import json

def get_issue_comments(repo, issue_number):
try:
# Requires gh CLI to be installed and authenticated
cmd = f"gh api repos/{repo}/issues/{issue_number}/comments"
result = subprocess.run(cmd, shell=True, capture_output=True, text=True, check=True)
comments_data = json.loads(result.stdout)
# Concatenate all comment bodies, separated for clarity
return "\n---\n".join([c['body'] for c in comments_data])
except subprocess.CalledProcessError as e:
print(f"Error fetching comments for issue {issue_number}: {e}")
print(e.stderr)
return None

This gives me a single string of all comments, nicely delimited, ready for the LLM.

2. Ollama & llama3:8b (My Local Workhorse)

I already had Ollama installed and llama3:8b downloaded. It's a fantastic local model that strikes a great balance between performance and size. Running it is as simple as ollama run llama3:8b in your terminal, then feeding it input.

3. The Magic: Prompt Engineering (Focus on Actionability)

My initial prompt was too generic, something like, "Summarize this GitHub issue comment thread." It gave me bland, academic recaps that weren't actionable. I needed intelligence that helped me triage, not just understand. After a few iterations, this is the prompt that clicked:

You are an expert software engineer and project manager. I am providing you with the entire comment history for a GitHub issue. Your task is to summarize the core problem, the proposed solutions, any blocking issues, and the clear next steps for triaging this issue. Be concise and focus on actionable insights for an engineering team.

{TEXT_TO_SUMMARIZE_HERE}

Summary:

That last Summary: at the end is crucial. It primes the model to immediately start its output with the summary itself, making parsing easier.

4. Tying It Together: A Python Script

I combined the comment fetching with a function to send the data to Ollama:

python
def summarize_with_ollama(text_to_summarize):
prompt = f"""You are an expert software engineer and project manager. I am providing you with the entire comment history for a GitHub issue. Your task is to summarize the core problem, the proposed solutions, any blocking issues, and the clear next steps for triaging this issue. Be concise and focus on actionable insights for an engineering team.

{text_to_summarize}

Summary:
"""
try:
# Using Popen for better control over stdin/stdout
cmd = ["ollama", "run", "llama3:8b"]
process = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True)
output, _ = process.communicate(prompt) # Send prompt, get output

    # Ollama often echoes the prompt; find where the actual response starts
    response_start_marker = "Summary:"
    response_start_index = output.find(response_start_marker)
    if response_start_index != -1:
        return output[response_start_index + len(response_start_marker):].strip()
    return output.strip() # Fallback if marker isn't found perfectly
except Exception as e:
    print(f"Error calling Ollama: {e}")
    return "Failed to get summary."

Example usage (put this in a main block or another script)

if name == "main":
REPO = "your-github-org/your-repo"
ISSUE_NUMBERS = [123, 456, 789] # Your backlog issues

for issue_num in ISSUE_NUMBERS:
    print(f"\n--- Summarizing Issue #{issue_num} ---")
    all_comments = get_issue_comments(REPO, issue_num)
    if all_comments:
        summary = summarize_with_ollama(all_comments)
        print(f"Issue #{issue_num} Summary:\n{summary}")
    else:
        print(f"Could not retrieve comments for Issue #{issue_num}.")

The Result: My Triage Superpower

The difference was night and day. A complex issue that would have taken me 15-20 minutes to fully grasp, I could now triage in under 5 minutes thanks to a solid, actionable summary. I probably saved myself at least an hour this week alone, and my team will definitely feel the benefit of more accurately triaged issues, since I'm not making snap judgments based on partial understanding.

It’s funny how often the simplest solutions, built on existing tools, turn out to be the most effective. No fancy SaaS, no API keys, just my local machine, Ollama, and a little bit of Python. If you're drowning in GitHub comments, I highly recommend giving a local LLM a shot. It's a game-changer for reclaiming your sprint.

My LLM-Powered Sidekick for Painless PR Descriptions

Nexus Labs — Mon, 02 Nov 2026 15:01:23 +0000

You know that feeling. You've just poured your soul into a feature, pushed your branch, and then... the dreaded blank PR description box. Your brain is fried. You just want to ship it, but you should write a good description. Explaining your changes, the 'why,' the 'what.' It's a mental tax I've always hated.

After one particularly grueling merge week, I decided enough was enough. What if an LLM could at least draft these things for me? I wanted something that wasn't just a generic recap but actually understood the context of my changes. So, over a recent weekend, I set out to build it. Here’s how it went down:

1. The "Duh" Moment & First Try

My initial thought was, 'Hey, LLMs are good at summarizing code, right?' What if I just fed it the raw git diff? Simple. Elegant. Probably naive. But that's usually where my best (and worst) ideas start. I figured the minimal effort approach was the best way to get started and see what broke.

2. Grabbing the Diff

Getting the diff programmatically is pretty straightforward. A quick subprocess.run call in Python did the trick. I usually compare against main for PRs, so it was git diff main...HEAD. The ... is crucial here to get all the commits on the current branch, not just the changes since the branch diverged from main.

python
import subprocess

def get_git_diff():
try:
# Compare current branch HEAD against main
result = subprocess.run(
['git', 'diff', 'main...HEAD'],
capture_output=True, text=True, check=True
)
return result.stdout
except subprocess.CalledProcessError as e:
print(f"Error getting diff: {e.stderr}")
return None

I was using Python 3.10 for this little script, just because that's what my pyenv default was set to.

3. Initial Prompting & The LLM

For the LLM, I picked gpt-3.5-turbo-0125 for its speed and cost-effectiveness for initial experiments. My first prompt was embarrassingly simple: Summarize these git changes into a pull request description, explaining what was changed and why. Keep it concise. Then I just appended the diff directly after that.

4. The Unexpected Hiccup (My First Disappointment)

This is where I ran into my first major roadblock. I was expecting magic, but I got... vanilla. The LLM would spit out descriptions that were technically correct, but completely lacked context. It would say, 'Added a new function to handle user authentication,' but it wouldn't know why that function was needed, or what problem it solved from a higher level.

It was too literal to the code, like a junior dev reading lines and rephrasing them. For example, if I had a feature branch that fixed a bug where users couldn't log in after a password reset, the LLM would say: 'Modified auth_service.py to update login_user function parameters.' No mention of 'bug fix' or 'password reset flow.' It was just a dry recap of the code changes. I'd spent about 2 hours on this initial setup, only to get something barely better than me just looking at the diff.

5. Injecting Context – The "Why"

I quickly realized the LLM only knows what you feed it. The git diff shows what changed, but not why. The 'why' often lives in commit messages, or maybe even in a Jira ticket ID if you're disciplined. I started experimenting with adding the commit messages from the current branch.

python
def get_git_log():
try:
# Get commit messages for current branch (since diverging from main)
result = subprocess.run(
['git', 'log', 'main...HEAD', '--pretty=format:%s%n%b'],
capture_output=True, text=True, check=True
)
return result.stdout
except subprocess.CalledProcessError as e:
print(f"Error getting log: {e.stderr}")
return None

Then I updated my prompt. Instead of just the diff, I added a section for Commit Messages and Git Diff and tweaked the prompt instruction: Analyze the following commit messages and git diff. Based on this information, generate a detailed and context-aware pull request description. Focus on the problem being solved, the solution implemented, and any relevant impact. Include a suggested title and a summary section. This made a huge difference. Suddenly, the LLM started connecting the dots between the commits and the code changes.

6. The Working Script and Beyond

Now I had something genuinely useful. I wrapped it all up in a simple Python script that takes a few arguments (like --clipboard to copy directly, or --print to just show). I added some markdown formatting to the LLM's output request, asking for sections like 'Title', 'Summary', 'Problem', 'Solution', and 'Changes'.

python

Simplified example of combining diff and log for LLM input

full_context = f"Commit Messages:\n{get_git_log()}\n\nGit Diff:\n{get_git_diff()}"
prompt = f"""
Analyze the following commit messages and git diff. Based on this information, generate a detailed and context-aware pull request description.
Focus on the problem being solved, the solution implemented, and any relevant impact.
Include a suggested title and the following sections:

Summary: A high-level overview of the changes.
Problem: What issue does this PR address?
Solution: How does this PR solve the problem?
Changes: A brief bulleted list of technical changes.

Here's the context:\n\n{full_context}
"""

Then send 'prompt' to LLM API and process the response

This little automation isn't going to write your entire codebase, but it's a huge boost to developer experience. It gets me 80% of the way there, and I just fill in the last 20% with specific details or design decisions. It’s wild how much difference adding those commit messages made. Next up, maybe integrating it directly into my editor or as a pre-commit hook that generates a draft PR description right before I push. No more blank boxes for me!

Sifting Customer Feedback: My No-Code AI Fix

Nexus Labs — Sun, 01 Nov 2026 15:00:44 +0000

Every Monday morning, it was the same grind. Our weekly customer survey would land, usually 70-100 responses, sometimes more. And each one was a potential goldmine of product insights – or a time sink. I’d try to read through them all, looking for patterns, specific feature requests, bug reports, or general sentiment. It took me a solid 3-4 hours, often more, to just skim and make some rough notes. By the time I synthesized anything meaningful, half the week was gone, and the actionable feedback felt less urgent. I knew there had to be a better way than just me, a cup of coffee, and a very long spreadsheet.

My first attempts were... basic. I’d dump everything into a Google Sheet and use CTRL+F for keywords like 'bug,' 'slow,' 'missing feature,' etc. It helped, but it was incredibly blunt. If someone said 'the app felt sluggish when loading images,' my 'slow' search would catch it, but it wouldn't tell me what was slow, or if it was a critical issue versus a minor annoyance. I even tried some simple Python scripting, thinking I could build a regex-based categorizer. I spent a whole afternoon wrestling with re.search and re.findall trying to identify phrases, but the moment I added a new category or a synonym, the script broke, or I missed half the relevant feedback. It felt like I was building a brittle house of cards. The script got so complex for anything beyond 3-4 basic keywords, it was clear I was trying to code a natural language understanding engine from scratch – a fool's errand for an indie dev on a tight schedule.

python

A snippet of my overly ambitious, ultimately failed Python script idea

def categorize_feedback(text):
if re.search(r'bug|error|glitch', text, re.IGNORECASE):
return "Bug Report"
if re.search(r'slow|lag|sluggish', text, re.IGNORECASE):
return "Performance Issue"
# ...and so on, but it quickly became unmanageable
return "General Feedback"

That script ended up in the digital graveyard after about 6 hours of futile effort. It just wasn't intelligent enough. I needed something that could understand the intent, not just match keywords.

That's when I started looking into no-code AI builders. I'd heard whispers about them, but always assumed they were for enterprise teams with massive datasets. Turns out, they're perfect for small-scale, repetitive text analysis tasks like mine. I settled on trying out a platform that specializes in text classification (think tools like Levity.ai or even some of the more advanced features coming to Airtable or Zapier/Make.com's AI modules).

The setup was surprisingly straightforward. I connected my Google Sheet where the survey responses land. Then, the core task: defining my categories. I brainstormed the key themes I always looked for: 'Bug Report,' 'Feature Request,' 'UX/UI Improvement,' 'Performance,' 'General Positive,' 'General Negative.'

The magic happened with the training. For each category, I provided 10-15 example sentences from past survey responses. For instance, under 'Bug Report,' I'd put:

'The login button doesn't work after updating.'
'Keeps crashing when I try to upload a photo.'
'There's a typo on the pricing page.'

I did this for all six categories. The platform then used these examples to train its underlying AI model. It took about 30 minutes to do the initial setup and training.

Once trained, I set up an automation: whenever a new response comes into the Google Sheet, the AI builder processes the 'Feedback Text' column, and then populates new columns I created: 'AI_Category,' 'AI_Sentiment,' and 'AI_Key_Phrases.'

The results were immediate and frankly, mind-blowing. That 3-4 hour Monday morning chore? It's now a 15-minute review. I glance at the 'AI_Category' column, filter for 'Bug Report' or 'Feature Request,' and boom – I have a concise list of actionable items. The AI even picks up nuances I might have missed when skimming. For example, it correctly categorized 'The new notification badge is a bit too bright, distracts me' as a 'UX/UI Improvement,' something my keyword searches would never have caught.

It’s not 100% perfect, of course. Sometimes it misclassifies a particularly ambiguous response, maybe 5% of the time. But even then, it's usually just a quick manual adjustment. The sheer productivity gain is immense. I'm now getting insights faster, prioritizing better, and spending my time actually building, not sifting through data. If you're drowning in qualitative feedback, seriously, look into a no-code AI text classification tool. It's a game-changer.

Copilot Chat Just Saved My Monday Release (and My Sanity) from Date Errors

Nexus Labs — Sat, 31 Oct 2026 14:00:41 +0000

Okay, so you know that feeling? The one where a deadline is breathing down your neck, and there's one bug, a seemingly simple one, that refuses to die. That was me, all last week, staring down a looming Monday release for my client's new analytics dashboard.

The Problem: My Dates Were Liars

My current project is a React app, pulling reporting data from a REST API. Dates from the API arrive as simple YYYY-MM-DD strings, like "2026-10-31". Simple, right? I needed to display these dates correctly in the UI and also use them for various filtering operations. My initial parsing logic was pretty straightforward, mostly relying on new Date(apiDateString) or sometimes a quick dayjs conversion.

But for some reason, these dates were consistently off by exactly one day. Sometimes they'd display as the previous day (2026-10-30), sometimes the next (2026-11-01). It wasn't random; it was like there was a hidden rule, a cruel joke, playing out differently based on specific dates or perhaps my local timezone. It drove me absolutely batty. The client wasn't going to be thrilled if their "October 31st" data showed up under "October 30th" or "November 1st".

What I Tried (And Failed At)

I probably spent a solid 5 hours over Friday afternoon and Saturday morning trying to pin this down. My usual bag of tricks felt useless:

Console logs everywhere: console.log('original:', apiDateString, 'parsed:', new Date(apiDateString)); Just confirmed what I already knew: the parsedDate was wrong. Often, if I was in America/New_York, new Date('2026-10-31') would resolve to Fri Oct 31 2026 00:00:00 GMT-0400 (Eastern Daylight Time), but when I tried to format it or use it in a dayjs comparison, it would sometimes internally shift, especially if it touched any UTC conversion logic.
VS Code Debugger: Stepped through the code. I could see new Date() doing its thing, but the exact moment of the +-1 day shift was like chasing smoke. It always seemed to happen just after my line, in some internal browser or Date object magic.
Library Swapping: "Maybe it's dayjs?" I thought. Swapped it out for moment.js (yes, I know, but I was desperate). Same issue. Swapped back to plain Date objects to simplify the problem space.
Stack Overflow: Scoured posts. Found a million variations of "new Date() and timezones" but nothing that perfectly matched my elusive off-by-one behavior across different scenarios.

My brain was completely fried. I was dreading having to tell the client we might miss the Monday deadline because of a stupid date bug.

What Actually Fixed It: Enter GitHub Copilot Chat

I was sitting there, head in my hands, when I remembered seeing some tweets about the new features in Copilot Chat – especially its ability to understand context better. I'm running VS Code v1.83.1 with Copilot Chat v0.10.1.

I pulled up the component with the problematic parsing, highlighted the relevant lines (which, honestly, were just new Date(apiDateString) and then some formatting calls), and opened the Copilot Chat panel. My first prompt was pretty generic:

"This date parsing logic new Date(apiDateString) is giving me dates that are off by one day. The apiDateString is YYYY-MM-DD from a UTC API. I need it to correctly represent that calendar date locally. What's wrong?"

Copilot thought for a second. Its initial suggestions were a bit too broad – check the API, ensure locale settings. Good generic advice, but not the specific bug.

I tried again, being more pointed:

"The YYYY-MM-DD string new Date('2026-10-31') is causing issues. When new Date() is called with only a date string (no time component), how does JavaScript handle the timezone and what's a robust way to ensure it represents October 31st in any local timezone?"

This is where it got interesting. Copilot's response pinpointed the exact ambiguity I was wrestling with. It explained that new Date('YYYY-MM-DD') is often interpreted as YYYY-MM-DDT00:00:00 in the local timezone. However, if that YYYY-MM-DD string is effectively coming from a UTC context, then displaying 00:00:00 local time might shift the calendar day if your local timezone is significantly offset from UTC (e.g., if you're GMT-5, T00:00:00 local time is T05:00:00Z – the previous calendar day in UTC terms). It's subtle but devastating for accuracy.

Copilot offered a few solutions, but the one that immediately clicked and made the most sense was explicitly constructing the date in UTC and then letting the display logic handle the local conversion, ensuring the calendar date was preserved:

javascript
// Original problematic code (simplified example)
const apiDateString = "2026-10-31";
const problematicDate = new Date(apiDateString); // Interpreted as Oct 31st 00:00:00 local time

console.log("Problematic display:", problematicDate.toLocaleDateString('en-US'));
// If local timezone is e.g. GMT-5, this might resolve to Oct 30th after some internal conversions

// Copilot's suggested fix: Construct in UTC
const [year, month, day] = apiDateString.split('-').map(Number);
const fixedDate = new Date(Date.UTC(year, month - 1, day)); // Month is 0-indexed!

console.log("Corrected display (forcing local interpretation of UTC date):");
// To display this UTC date string as the intended local calendar date:
console.log(new Date(fixedDate.getUTCFullYear(), fixedDate.getUTCMonth(), fixedDate.getUTCDate()).toLocaleDateString('en-US'));

// A slightly simpler approach Copilot also offered, if you trust your local system to interpret T00:00:00 correctly:
const alternativeFixedDate = new Date(apiDateString + 'T00:00:00'); // Explicitly set time to midnight local
console.log("Alternative fixed display:", alternativeFixedDate.toLocaleDateString('en-US'));

The Date.UTC approach was exactly what I needed. By explicitly creating the date in UTC from the YYYY-MM-DD components, I ensured that 2026-10-31 meant 2026-10-31 UTC, without my local timezone meddling with the calendar day itself. When I then displayed it, it correctly translated to the local time, but retaining the October 31st part.

Within 15 minutes of chatting with Copilot, I had a working solution. My dayjs logic was refactored similarly. The deadline is saved, and my sanity is (mostly) intact. I'm still blown away by how it identified the nuance of new Date()'s behavior and offered such a precise fix. It's not just a fancy autocomplete anymore; it's a legitimate debugging partner.

Now, if you'll excuse me, I need a coffee. And maybe to update my README.md to thank Copilot. 😂

No More Phantom Columns: Finally Getting LLM-Generated SQL to Behave

Nexus Labs — Thu, 29 Oct 2026 14:01:02 +0000

Okay, so picture this: we're rolling out a brand new reporting schema for our platform. Shiny, optimized, a little different from the old one. My grand vision? Let our team ask questions in plain English and have an LLM spit out the SQL. No more waiting for dev cycles just to get a COUNT(*) from a new table.

Sounds great on paper, right? The promise of text-to-SQL is seductive. I mean, it can generate SQL. Give it a schema, ask a question, and boom, query. But my early experiments quickly turned into a battle against phantom columns.

The Problem: Ghost in the Machine (or, in the Schema)

The LLM, bless its creative heart, kept making stuff up. I'd feed it our beautiful, meticulously designed CREATE TABLE statements for, say, our user_events table with event_id, user_id, event_type, event_timestamp. Then I'd ask it, "Show me the last 10 'login' events for user_id 123." And it'd give me back something like:

sql
SELECT * FROM user_events
WHERE user_identifier = 123 AND event_name = 'login'
ORDER BY created_at DESC LIMIT 10;

See the issue? user_identifier? event_name? created_at? None of those exist in my schema. It was close enough to be frustrating, but just wrong enough to crash hard with psycopg2.errors.UndefinedColumn: column "user_identifier" does not exist every single time. It was like it had seen similar schemas in its training data and decided to just invent columns that felt right, even when given explicit instructions.

This wasn't just a minor annoyance; it completely undermined the reliability of the system. I couldn't trust anything it generated, and manual validation defeated the whole purpose.

What I Tried (and Mostly Failed At)

I spent a good three days banging my head against this. My initial openai client was 1.3.0, and I even upgraded to 1.10.0 thinking some magic might happen there. It didn't. The problem wasn't the library; it was my approach.

Dumping the Entire Schema: My first thought was, "More context equals better results, right?" So I serialized our entire reporting schema (which has about 47 tables now) into a massive string and shoved it into the prompt. The LLM either got confused, missed crucial details, or the prompt simply became too long for gpt-3.5-turbo to handle effectively. gpt-4 was better, but prohibitively slow and expensive for every single query.
Aggressive System Prompts: I tried everything. "ONLY use column names from the provided schema." "DO NOT invent columns." "STICK STRICTLY to the schema definitions." I even tried putting it in all caps and using negative prompting ("do not use..."). It felt like talking to a very polite, very helpful, but occasionally hallucinating assistant.
Few-Shot Examples: I gave it a dozen good (natural language, correct SQL) pairs based on our new schema. This helped for queries very similar to the examples. But as soon as a new pattern emerged or a slightly different column was needed, it would still slip into the phantom column trap. Maintaining enough examples to cover all bases felt like writing the SQL myself again, which, again, defeated the purpose.

The Breakthrough: Contextual Tools and Tighter Guardrails

The real game-changer wasn't one silver bullet, but a combination of focused context provision and leveraging the LLM's function-calling capabilities. It became a multi-stage process, less about magic and more about engineering.

1. Smart Schema Snippet Generation

Instead of the whole schema, I realized the LLM only needed the relevant parts. This was critical for our new schema, which the LLM had no prior training on.

I built a small Python utility that, given a natural language query, would do a quick keyword scan (and eventually, I'll layer in semantic search using embeddings) over our custom schema metadata. This metadata isn't just CREATE TABLE statements; it includes human-readable descriptions for tables and columns too.

For example, if the user asks "How many active subscriptions do we have in October?", my utility identifies subscriptions and users tables (because active might relate to user status, though subscriptions holds the core info). Then, it dynamically generates a concise DDL snippet only for those tables, like this:

sql
-- Table: subscriptions
-- Details about user subscriptions.
CREATE TABLE subscriptions (
subscription_id UUID PRIMARY KEY,
user_id UUID NOT NULL REFERENCES users(user_id),
plan_type VARCHAR(50) NOT NULL, -- e.g., 'monthly', 'annual'
start_date DATE NOT NULL,
end_date DATE
);

-- Table: users
-- Information about registered users.
CREATE TABLE users (
user_id UUID PRIMARY KEY,
first_name VARCHAR(100),
last_name VARCHAR(100),
signup_date DATE NOT NULL
);

This focused context greatly reduced the LLM's opportunity to hallucinate.

2. Function Calling for Strict SQL Output

This was the biggest win. Instead of asking the LLM to just return SQL as raw text, I defined a tool for it. The LLM's job became to call a function generate_sql_query with specific arguments.

python
tools = [
{
"type": "function",
"function": {
"name": "generate_sql_query",
"description": "Generates a PostgreSQL query to answer a natural language question based on the provided schema.",
"parameters": {
"type": "object",
"properties": {
"sql_query": {
"type": "string",
"description": "The PostgreSQL query."
},
"reasoning": {
"type": "string",
"description": "Brief explanation of the query's logic."
}
},
"required": ["sql_query", "reasoning"],
},
}
}
]

The system prompt was then refined: "You must use the generate_sql_query tool. The sql_query argument must only use column names present in the provided schema snippet. If a column is not explicitly in the snippet, you cannot use it." This explicit constraint, enforced by the structured nature of function calls, felt like putting the LLM in a very specific, well-defined box, limiting its creative freedom where it mattered most.

3. The Feedback Loop (Error Handling)

Even with all these guardrails, occasionally a phantom column would slip through (maybe one time out of twenty, down from ten out of ten). My application catches the psycopg2.errors.UndefinedColumn error during execution. I then feed that exact error message back to the LLM in the next turn of the conversation, along with a prompt to correct its previous output. This automated retry and self-correction mechanism made the system robust.

The Takeaway

It wasn't about a more powerful LLM model, but about smarter interaction design. The LLM is an incredible reasoning engine, but it needs clear, concise, and relevant context, alongside strict guardrails on its output. It took me a solid week of iterating, trying different models, prompt structures, and context injection methods, but now, finally, our reporting dashboard can reliably generate SQL for our new schema. It feels pretty great.

Now, to make that schema selector even smarter... 🤖

Taming the Ticket Queue: My GPT-4 + Zapier Automation Story

Nexus Labs — Wed, 28 Oct 2026 14:00:34 +0000

Remember those days? The support inbox would swell up, and every morning started with a game of 'who gets this one?' It was a constant mental load, manually sifting through incoming customer support tickets, trying to figure out if it was a frontend bug, a database issue, a simple billing question, or yet another ambitious feature request. Not only was it a huge time sink for me, but tickets often sat in a general queue longer than they should have, or worse, got assigned to the wrong team, leading to unnecessary re-assignments and frustration.

The Problem: Manual Triage Was Killing Me (and Our Response Times)

As our user base grew, so did the volume of support requests. We're a small team, and every minute counts. Having me or another developer spend an hour each day just triaging tickets was unsustainable. It felt like we were always playing catch-up, and our engineers were getting bombarded with tickets outside their domain, which meant context switching and delays.

My primary goal was to get tickets to the right engineer or team as quickly as possible, with minimal human intervention. We use Intercom for customer support, but its native automation, while decent for simple keyword matching, just wasn't cutting it for the nuanced language of customer problems.

What I Tried: Rules, Regex, and Almost a Custom NLP Model

Initially, I tried to lean heavily on Intercom's existing rules engine. I spent what felt like weeks writing complex IF this AND NOT that OR this OTHER thing rules. I tried keyword matching, regular expressions for common error codes, and even some basic sentiment analysis. It was an absolute nightmare.

Keyword matching: Too brittle. A user describing a "login issue" could mean anything from a forgotten password (billing/account team) to an API authentication failure (backend team). Keywords alone lacked context.
Complex Rules: Maintenance was a full-time job. Every time a new feature shipped, or a common issue surfaced, I had to update a sprawling web of rules. It became an unmanageable mess. I even considered writing a small Python script that would hit the Intercom API, process the text with some open-source NLP library, and then update the ticket. But honestly, the thought of maintaining another microservice, setting up hosting, and dealing with model training and drift just for ticket routing made me want to curl up and cry. I'm an indie dev; I need to ship, not build infrastructure for internal tooling that doesn't directly serve customers.

What Actually Fixed It: GPT-4 + Zapier to the Rescue

Then, one evening, while messing around with the OpenAI API for another project, it clicked. GPT-4, with its incredible language understanding, could absolutely solve this. And better yet, I could probably stitch it together with Zapier, avoiding any custom code deployments.

Here’s how I built the automation, step-by-step:

The Trigger: A new conversation starts in Intercom. This was straightforward with Zapier's built-in Intercom integration. When a new inbound message lands, Zapier wakes up.
The GPT-4 Action: This is where the magic happens. I used Zapier's OpenAI action (specifically, Send Conversation). The key here is crafting a solid prompt. I spent a good half-day iterating on this, burning through about $7 worth of API credits until it consistently spat out what I needed. Early versions sometimes hallucinated categories or struggled with ambiguity, so I had to be very explicit.

Here's a simplified version of my current prompt, tweaked for clarity:

You are a highly experienced customer support triage agent for a SaaS company. Your goal is to accurately categorize and assign incoming support tickets. Read the customer's initial message and determine:
1. Category: One of the following: "Billing & Account", "Bug Report", "Feature Request", "Technical Issue", "General Inquiry", "Onboarding".
2. Assigned Team: One of the following: "Frontend Team", "Backend Team", "Infrastructure Team", "Mobile Team", "Product Team", "Success Team".
3. Priority: "High", "Medium", "Low". Default to "Medium" unless critical (e.g., service outage, data loss).
Output your response STRICTLY as a JSON object with 'category', 'assigned_team', and 'priority' keys. DO NOT include any other text.

Customer Message:

I then passed the {{intercom_conversation_first_message}} into the prompt. The model was gpt-4-turbo-preview initially, now just gpt-4-turbo. The output from GPT-4 is a neat JSON object like {"category": "Bug Report", "assigned_team": "Backend Team", "priority": "High"}.
The Parser (Sometimes): Occasionally, GPT-4 might include an apology or stray word outside the JSON. I added a Code by Zapier step that simply takes the raw text output, trims whitespace, and then runs JSON.parse() on it. This ensures clean data for the next step. If it fails, I get an error notification, which helps me tweak the prompt.
Update Intercom Conversation: Finally, I use another Intercom action to Update Conversation. I map the category from the GPT-4 output to a custom attribute in Intercom called Ticket Category. More importantly, I map the assigned_team to the Intercom Assign to team field. I also set the Priority.

Now, when a new ticket comes in, within seconds, it's not only categorized but also assigned to the correct team, showing up in their dedicated inbox in Intercom. This has been a game-changer. Our average first response time has plummeted, and more importantly, our engineers are spending less time re-routing tickets and more time solving actual problems. It's not 100% perfect – about 95% of tickets are routed perfectly – but that remaining 5% is easily handled with a quick manual adjustment, which is a massive improvement over the previous chaos. This simple automation has probably saved us 10-15 hours a week across the team. Highly recommend giving it a shot if you're drowning in support tickets!

Make.com Dropping Large Email Attachments? Here's How I Fixed It.

Nexus Labs — Tue, 27 Oct 2026 14:00:41 +0000

Setting up a new automation in Make.com usually feels like magic. You connect a few blocks, map some fields, and boom – things just work. This time, though, I hit a wall with something that should have been simple: taking email attachments and dropping them into Google Drive.

The Ghosting Problem: Attachments Vanish

My goal was straightforward: whenever I received an email with an attachment in a specific Gmail inbox, I wanted that attachment automatically uploaded to a designated folder in Google Drive. For small files – think PDFs, .docx files, or tiny images under 10MB – it was flawless. The automation hummed along, and files appeared exactly where they should. Beautiful.

But then came the larger files. Design assets, client videos, zipped archives, anything over, say, 20-25MB. My Make.com scenario would run, show "success" in the execution logs, and confidently declare it had processed everything. Yet, when I went to Google Drive, those larger files were nowhere to be found. Not in the target folder, not in the root, not even in the trash. They simply vanished into the digital ether. Pure ghosting.

This wasn't just annoying; it was a critical failure for a workflow I was relying on. The irony of a "no-code" tool demanding hours of debugging was not lost on me.

What I Tried (and what didn't work)

I spent a good 6-7 hours over two evenings trying to understand what was going on. Here’s the rundown of my troubleshooting journey:

Deep Dive into Make.com Logs: My first instinct was to check the execution history. I meticulously opened each successful run, expanded every module, and examined the input/output bundles. Every step was green. The Gmail - Get an Attachment module reported that it had successfully retrieved the attachment, even showing the correct fileSize for the missing large files. The Google Drive - Upload a File module also showed a green checkmark. This was infuriating. No errors, just no file.
Google Drive Sanity Checks: I double-checked the target folder ID, browsed other folders, and yes, even scoured the Google Drive trash. Nothing. It wasn't a permissions issue either, as smaller files were happily landing in the exact same spot.
File Size Limits Research: I dug into Make.com's documentation for any hidden global file size limits or module-specific constraints. I also checked Google Drive API limits. While both have limits, they were far beyond the 20-25MB range where my issues started. Nothing immediately pointed to a hard wall I was hitting.
Module Reconfiguration: I deleted and re-added both the Gmail - Get an Attachment and Google Drive - Upload a File modules. I tried different ways of mapping the file data – sometimes Make.com can be particular about whether it expects a buffer or a URL or a specific file object. No combination seemed to make a difference.
Different Email Sources: Just in case it was a specific email client's fault, I sent large attachments from Outlook, ProtonMail, and even another Gmail account. The result was consistently the same: small files arrived, large files disappeared.

At this point, I was about ready to give up and write a custom script, which defeated the whole purpose of using a no-code platform.

The "Aha!" Moment: Explicit HTTP Download

My breakthrough came when I started looking very closely at the Gmail - Get an Attachment module's output bundle. While it provides the data (the file content), it also provides a webContentLink – a temporary, authenticated URL where the attachment can be directly downloaded.

It hit me: maybe the direct data transfer from the Gmail - Get an Attachment module to the Google Drive - Upload a File module wasn't as robust for large files as it appeared. Perhaps Make.com's internal handling of that abstract data object had a hidden timeout or memory buffer limitation when dealing with substantial binary payloads.

The Fix:

I inserted an HTTP - Get a File module between my Gmail - Get an Attachment and Google Drive - Upload a File modules. Instead of passing the data directly from the Gmail module to Google Drive, I did this:

Gmail - Get an Attachment: This module still runs first, retrieving the attachment details.
HTTP - Get a File: In this module, I used the webContentLink from the Gmail - Get an Attachment module as the URL. This explicitly tells Make.com: "Go download this file again via a standard HTTP request."
Google Drive - Upload a File: Now, I mapped the Data output from the HTTP - Get a File module to the File field in Google Drive.

It looked something like this:

[
{
"name": "Gmail - Get an Attachment",
"outputs": {
"webContentLink": "https://docs.google.com/a/example.com/uc?id=XYZ&export=download",
"filename": "MyLargeFile.zip",
"fileSize": 100000000,
"mimeType": "application/zip"
// ... other outputs
}
},
{
"name": "HTTP - Get a File",
"inputs": {
"url": "{{1.webContentLink}}" // Mapped from previous Gmail module
},
"outputs": {
"data": "",
"filename": "MyLargeFile.zip"
}
},
{
"name": "Google Drive - Upload a File",
"inputs": {
"folderId": "12345",
"file": {
"data": "{{2.data}}", // Mapped from HTTP module
"filename": "{{1.filename}}"
}
}
}
]

This small change made all the difference. The HTTP - Get a File module seemed to handle the large file transfer much more reliably. I tested it with a 100MB video file, and boom – it appeared in Google Drive instantly. Every large attachment since then has worked perfectly.

It seems that while the Gmail - Get an Attachment module can get the file details, for truly large files, explicitly downloading the content via HTTP - Get a File before passing it to the next module is the more robust approach. Make.com, for all its magic, sometimes needs a little nudge to handle the heavy lifting just right. Hopefully, this saves you a few hours of head-banging!

Finally, Release Notes That Don't Suck: Automating with LLMs and GitHub Actions

Nexus Labs — Mon, 26 Oct 2026 14:00:39 +0000

You know that feeling. It’s Friday afternoon, you’ve just merged a bunch of features and fixes, and now it’s time for the dreaded release notes. You scroll through Git history, try to remember what each commit actually did, then cross-reference Jira to get the full picture. It's tedious, error-prone, and honestly, a soul-crushing task that pulls me away from the fun stuff. For years, I've just… avoided it, or penned the most generic 'bug fixes and improvements' ever.

The Problem: Manual Release Notes Are a Grind

I’m an indie developer. Every minute I spend on administrative overhead is a minute I’m not building. Manual release note generation was a huge time sink. My releases often had a dozen or more tickets, plus countless small commits. Synthesizing all that into something concise, accurate, and actually useful for users felt like pulling teeth. The worst part? Half the time, the notes were boilerplate anyway because I was too lazy (or busy) to dig deep. I knew there had to be a better way.

What I Tried First: Just Feed it to an LLM

My initial thought was pretty simple: large language models are good at summarizing, right? So, I wrote a quick Python script to grab git log --pretty=format:"%s" --since="last_release_tag" and fed the raw output to OpenAI's gpt-3.5-turbo. I figured it would just magically spit out perfect release notes.

It did not. It was a disaster, actually. The output was either:

Too generic: "This release includes various updates and bug fixes to improve application stability." (Yeah, thanks, I already said that.)
Too verbose: It would often just rephrase every single commit message, leading to paragraphs of developer-speak nobody cared about.
Missing context: My commit messages aren't always perfect. "Fix layout" doesn't tell a user which layout or why it was broken. Crucial context from Jira tickets was completely absent.

I tried refining the prompt, telling it to focus on user-facing changes, but without the Jira context, it was essentially guessing. The LLM didn't know if refactor: db migrations was an internal chore or a foundational change enabling a new feature. It needed more.

What Actually Fixed It: Context is King

After a few frustrating hours – probably half a day, just wrestling with the initial approach – I realized the LLM needed more structured, meaningful data. It needed the why behind the what. That meant integrating Jira. Here’s the pipeline I built and wrapped in a GitHub Action:

1. Grab Git Commits and Extract Jira IDs

First, I needed the commit messages since the last release. I use conventional commits (e.g., feat: PROJECT-123 Add user profiles), which made extracting Jira IDs a breeze. I used git log --pretty=format:"%s" "$(git describe --tags --abbrev=0)..HEAD" to get messages between the last tag and HEAD.

Then, a simple regex on the commit messages to find [A-Z]{2,}-\d+ patterns. This gave me a unique list of Jira tickets relevant to the changes.

2. Fetch Jira Details

For each unique Jira ID, I hit the Jira API. I’d previously done this with the jira-python library for local scripts, but for the GitHub Action, a curl call with secrets.JIRA_API_TOKEN felt more straightforward and less dependency-heavy. I fetched the ticket summary, type (Bug, Story, Task), and status. This provided the crucial context.

3. Combine and Prompt the LLM

This was the core. I structured the input for the LLM like this:

text
Here are recent Git commit messages:

feat: PROJECT-123 Add user profiles
fix: PROJECT-456 Improve dashboard load time
chore: internal cleanup

Here are the details for relevant Jira tickets:

PROJECT-123 (Story): Implement user profile management
PROJECT-456 (Bug): Dashboard slow on large datasets

Generate concise, user-facing release notes (3-5 bullet points). Focus on new features, bug fixes, and performance improvements. Exclude internal changes. Group similar items.

I found gpt-3.5-turbo-16k was perfectly capable and significantly faster/cheaper than gpt-4 for this task, costing me about $0.05-$0.10 per run. The 16k context window was useful for longer release cycles with many commits.

4. Wrap it in a GitHub Action

I created a Python script that performs steps 1-3, and then wrapped it in a GitHub Action. The action triggers on workflow_dispatch for manual runs or push when a new tag is created. It outputs the generated notes to a file and also as an action output, which can then be used by other steps (e.g., creating a GitHub release draft).

Here's a simplified snippet of the action YAML:

yaml
name: Generate Release Notes
on:
workflow_dispatch:
push:
tags:
- 'v*..'
jobs:
generate_notes:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install dependencies
run: pip install requests # For Jira API
- name: Run Release Note Generator
id: release_notes
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
JIRA_API_TOKEN: ${{ secrets.JIRA_API_TOKEN }}
JIRA_EMAIL: ${{ secrets.JIRA_EMAIL }}
JIRA_BASE_URL: ${{ secrets.JIRA_BASE_URL }}
run: python .github/scripts/generate_release_notes.py
- name: Create Release Draft
uses: softprops/action-gh-release@v1
if: startsWith(github.ref, 'refs/tags/')
with:
tag_name: ${{ github.ref_name }}
body_path: RELEASE_NOTES.md # Output from the script
draft: true

The Payoff: Concise, Accurate, Automated

Now, generating release notes is no longer a chore. I push a tag, the GitHub Action runs, and I get a draft release on GitHub with beautifully summarized notes, cross-referencing both Git and Jira. They're concise, user-focused, and crucially, they reflect what actually changed from a user's perspective. It took some elbow grease to set up, especially getting the prompt just right and handling Jira API authentication securely in the action, but it’s paid off immensely in saved time and mental overhead. My releases are smoother, and my users get better, more informative updates. It feels good to finally tame that beast.

Wrestling Action Items from LLMs: A Prompt Engineering Saga

Nexus Labs — Sat, 24 Oct 2026 14:00:37 +0000

Okay, so let's talk about something that's been a real pain point in my indie dev life: keeping project trackers updated from messy, unstructured meeting notes. You know the drill – someone scribbles down 'John to check on the database migration next week' or 'Team needs to decide on UI component library' and then it's a manual, tedious process to translate that into a Jira ticket or Trello card with a clear owner and deadline.

I probably don't need to tell you how much of a time sink this is, not to mention the errors and forgotten tasks. My project tracker was always lagging, and I was perpetually annoyed. Enter the dream: an LLM that could just read our meeting notes and spit out perfectly formatted action items with owners, ready to sync. Easy, right? Spoiler: It was anything but.

The Problem: When 'Smart' Isn't Smart Enough

The core issue wasn't just extracting text; it was extracting precise, actionable, owner-attributed text from human-speak. People talk in implied contexts, vague timelines, and often don't explicitly assign ownership. A typical note might say: "Need to investigate CDN options." Who's 'need to'? Everyone? No one? My LLM was constantly struggling with this ambiguity.

What I Tried (and Why It Mostly Tanked)

My initial approach was pretty naive, I'll admit. I thought a simple prompt like, "Extract all action items and their owners from the following meeting notes," would do the trick. It didn't. I'd get back things like:

Action: "Need to investigate CDN options." Owner: "(not specified)" (or worse, it would hallucinate an owner)
Action: "Database migration." Owner: "John to check on." (This is an action AND an owner mixed, not clean)

Attempt #1: Just ask nicely. Failed on ambiguity, nuance, and implied context. The LLM would often miss things, or include non-actionable chatter.

Attempt #2: Force a JSON structure. This felt like progress! I'd tell it, "Output a JSON array of objects, each with an 'action' and 'owner' field." I'd get the structure, but the content quality didn't magically improve. I was pulling my hair out when gpt-4-turbo would reliably output owner: null for things like 'John to follow up on API docs' despite John being right there, clear as day. The data was structured, but still unreliable.

Attempt #3: Few-shot prompting. I gave it a few perfect examples of raw notes mapped to ideal JSON output. This helped a little, especially with consistency in formatting, but it didn't solve the core problem of interpretation and reliable extraction across a wider variety of notes. It still struggled with edge cases and creative human language.

Attempt #4: Different LLMs. I tried gpt-3.5-turbo (too flaky), llama2:7b (even flakier locally via Ollama), and finally settled on gpt-4-turbo as the best of the bunch. While gpt-4-turbo was definitely superior, it still wasn't the magic bullet I hoped for. The fundamental problem wasn't the model's intelligence, but how I was asking it to apply it.

I probably easily sunk 10-12 hours into just these prompt engineering iterations, feeling like I was getting nowhere fast.

What Actually Fixed It: The Deconstruct and Validate Strategy

The breakthrough came when I realized I was asking the LLM to do too much in one go. Interpreting ambiguous human language, deciding what's actually an action item, figuring out the owner, and then formatting it perfectly – that's a lot of distinct cognitive steps. LLMs are powerful, but they shine when you break down complex tasks into simpler, sequential steps.

Here's the multi-stage approach that finally worked, turning my ~50% reliability into 90%+:

Step 1: Broad Extraction of Candidates

First, I prompt the LLM to just identify potential action items, without worrying too much about perfect phrasing or ownership yet. The goal here is to cast a wide net.

prompt
You are an assistant tasked with identifying potential action items from meeting notes. Read the following notes and list any sentence or phrase that might represent a task, decision, or follow-up required. Do not try to assign owners or rephrase yet, just list the raw potential items. Separate each item with a newline.

Meeting Notes:
"""
[My messy meeting notes text here]
"""

This gives me a list of raw strings like "John to check on the database migration next week." or "Team needs to decide on UI component library."

Step 2: Per-Item Validation and Refinement

This is the crucial step. For each candidate item from Step 1, I make a separate, focused call to the LLM. I give it the potential item and the original full meeting notes as context. This allows the LLM to focus on just one decision point at a time.

prompt
You are an assistant validating potential action items. Given a potential action item and the full meeting notes for context, determine if it is a concrete, actionable task with a clear owner.

If it IS actionable and has a clear owner:

Rephrase the action concisely and clearly.
Identify the specific owner. If multiple people are mentioned, pick the primary one responsible.
Output as a JSON object: {"action": "[concise action]", "owner": "[owner's name]"}

If it IS NOT actionable, lacks a clear owner, or is a duplicate/redundant:

Output: {"action": null, "owner": null}

Full Meeting Notes (for context):
"""
[My messy meeting notes text here]
"""

Potential Action Item to Validate:
"""
[Single candidate item from Step 1 here]
"""

This is where the magic happens. By giving it the full context again for each individual item, it can better infer ownership, clarify vague actions, or correctly identify non-actionable statements.

Step 3: Final Aggregation

Finally, I just filter out any {"action": null, "owner": null} results from Step 2 and consolidate the valid JSON objects into a single array. This gives me a clean, structured list of action items.

The Takeaway

This multi-stage approach bumped the processing time from around 3 seconds for a single gpt-4-turbo call to about 7-8 seconds (due to multiple API calls for potentially many candidate items), but the jump in reliability was absolutely worth it. It’s not about making the LLM 'smarter,' but about guiding it through a logical reasoning process. Deconstruct complex problems, validate decisions iteratively, and provide context where it's most needed. Sometimes, the 'simple' solution is actually a series of simple steps, not one giant leap.

I'm now much closer to my dream of fully automated project tracking. It's still not 100% perfect, but it's a hell of a lot better than my human brain trying to decipher scribbled notes at 5 PM on a Friday. And that, my friends, is a win in my book. 💪

Cracking PDF Catalogs: AI to Airtable for SKUs and Prices

Nexus Labs — Fri, 23 Oct 2026 14:00:51 +0000

Okay, so here's a problem that's probably familiar to anyone dealing with physical goods and a bunch of different suppliers: vendor PDF catalogs. For ages, these things were a huge pain point in my small e-commerce operation. We'd get dozens of them – some slick, some scanned, some just plain ugly. Each one was a unique snowflake of inconsistent formatting, and my team's job was to manually pull out SKUs, product descriptions, and prices into our Airtable database.

It was a grind. Seriously, soul-crushing. We're talking hours spent copying, pasting, correcting typos, and always, always being behind on updates. We needed a reliable way to automate this, desperately.

What I Tried (and What Broke)

My first thought was, "Python to the rescue!" I've always been a fan of scripting tedious tasks. I dove into libraries like pdfplumber and PyPDF2 (before pypdf was the main player). The idea was simple: extract text, find patterns, and get the data.

python
import pdfplumber

This often just gave me jumbled text or failed on complex layouts

with pdfplumber.open("catalog.pdf") as pdf:
for page in pdf.pages:
text = page.extract_text()
tables = page.extract_tables() # This was the real heartbreak
# ...then regex magic to try and find SKUs/prices

It worked for the most consistent, well-structured PDFs – maybe 20% of our inbound catalogs. But the moment a vendor switched from a neat table to text laid out in columns, or embedded product data within images, my scripts completely fell apart. I spent two solid days trying to craft robust regex patterns for each vendor. It was a constant game of whack-a-mole. The error message Table extraction failed for page X: could not find any explicit or implicit lines became a familiar, unwelcome friend.

I even briefly considered converting PDFs to images and running OCR, but the quality wasn't consistent enough for reliable data extraction across all the weird fonts and layouts we encountered. Then there were the template-based SaaS tools. They were better, requiring me to define a 'template' for each PDF layout. But again, a new vendor or a catalog redesign meant more manual setup and a subscription cost that added up fast for an indie operation.

What Actually Fixed It: Unleashing the LLM

I'd been playing around with Large Language Models (LLMs) for other tasks and thought, "Could an LLM just read the PDF and tell me the SKU, price, and description?" The answer, after some serious tinkering, was a resounding yes, but not directly with the PDF itself.

Here's the approach that finally clicked:

Extracting Raw Text: Forget pdfplumber for table parsing. The simplest, most reliable first step was to just get all the visible text out of the PDF. pypdf (the modern, maintained version of PyPDF2) is great for this.

python
import pypdf

reader = pypdf.PdfReader("vendor_catalog.pdf")
text_content = ""
for page in reader.pages:
text_content += page.extract_text() + "\n\n" # Add extra newlines for separation

This gives you a big, often messy, string of text representing the entire document.
Prompting the LLM: This was the crucial part. I fed this raw, often unstructured, text into an LLM. I'm using a self-hosted llama-3-70b-instruct instance via Ollama – saved me a ton on API costs. The prompt needs to be extremely clear and specify the desired output format.

You are an expert at extracting product information from vendor catalogs.
From the following text, extract the SKU, product description, and price.
The product description should be concise and accurate.
The price should include the currency symbol and be in a consistent format (e.g., $XX.XX).
If multiple products are listed, extract details for EACH product, even if not in a table.
If you cannot find a specific piece of information, respond with "N/A".
Respond in JSON format as an array of objects.

Example Output:
[
{"sku": "ABC-123", "description": "Widget Deluxe", "price": "$19.99"},
{"sku": "XYZ-456", "description": "Gadget Pro", "price": "$299.00"}
]

--- START CATALOG TEXT ---
{text_content}
--- END CATALOG TEXT ---

This worked like magic. The LLM, with its understanding of language and context, was able to parse the varying layouts, infer relationships between product names, SKUs, and prices, and output a structured JSON list. It wasn't 100% perfect on the first pass – maybe 90% accuracy – but good enough for a quick human review and correction.
Airtable Integration: From there, it was straightforward. A simple Python script takes the JSON output and uses the pyairtable library to push the extracted products into our Airtable base. I added a Review Status field that defaults to 'Needs Review' for new AI-extracted entries, making the human verification step super clear.

The Payoff

What used to be a half-day ordeal for a complex 20-page catalog now takes about 30 seconds per PDF, end-to-end. My team is no longer dreading catalog updates. This approach handles wildly different layouts, even from new vendors, without me needing to rewrite regex or define new templates. It’s effectively 'set it and forget it' for the majority of our PDFs.

I'm still refining the prompt, maybe adding more few-shot examples for edge cases, but honestly, this felt like finally cracking a problem that had been nagging me for years. It's a huge boost to our productivity and just makes life easier.

The Fight for Automated Standup Summaries (and How I Won)

Nexus Labs — Thu, 22 Oct 2026 14:00:32 +0000

Alright, fellow developers and automation enthusiasts, let me tell you about my recent wrestling match with Zapier. Our team, like many, has a daily 15-minute Google Meet standup. Crucial for alignment, but frankly, the post-meeting ritual was a mess. Someone would try to jot down notes, action items would get lost, and summaries were... well, non-existent. My mission: automate the whole darn thing – transcribe, summarize, and pluck out those elusive action items directly into Notion.

The Problem: Manual Labor and Missing Memos

Imagine this: five standups a week. That's five times someone needs to actively listen, take notes, then synthesize it into a coherent summary, and finally, manually add action items to our Notion task board. It wasn't sustainable. Action items were consistently slipping through the cracks, and having a searchable record of what was discussed was a pipe dream. I needed a reliable way to get the audio, feed it to an AI, and then push the results to Notion. Seemed straightforward, right? Narrator: It was not.

What I Tried: A Series of Unfortunate Zaps

My first thought was, naturally, Zapier. It's the glue for so many of my side projects. The initial idea was simple: Google Meet records to Google Drive, Zapier detects a new file, sends it to an AI transcriber, then pushes to Notion. What could go wrong?

Attempt 1: Google Drive Trigger Hell. I set up a Zap to trigger on a 'New File in Folder' in Google Drive. This immediately hit a wall. Google Meet recordings don't always appear instantly. Sometimes they took 10 minutes, sometimes an hour. Zapier's polling sometimes missed them entirely, or triggered before the file was fully processed by Google. About 20% of our standups just… vanished into the ether. We even saw a baffling File not found error in Zapier after a successful trigger, presumably because Google Drive hadn't quite settled its internal state. Frustrating.

Attempt 2: Custom Transcriber via Webhook. Okay, so Google Drive wasn't cutting it. What if I recorded locally and uploaded? Too much manual work, defeating the automation goal. I looked at various transcription APIs directly – AssemblyAI, OpenAI's Whisper. I even wrote a quick Python script to upload a file to Whisper and get a transcript back. This worked, but the problem remained: how do I get the audio reliably from the Google Meet into my script's hands without manual intervention? This felt like building half a house when I just needed a specific type of window.

Attempt 3: Generic AI Summaries and Action Item Extraction. Once I occasionally got a transcript, I piped it into an OpenAI 'Send Prompt' action. My initial prompts were laughably bad. Something like, "Summarize this meeting transcript and list action items." The summaries were often too generic, and action items were hit-or-miss. Sometimes it would list things that weren't actions at all, or miss clear ones. It became clear that what I asked the AI was as important as how I got the data to it.

I spent a solid eight hours over two days just trying to get the initial audio input reliable, not to mention tweaking prompts. The mental overhead was draining.

What Actually Fixed It: The Right Tool and a Surgical Prompt

The breakthrough came when I shifted focus from how to get the audio to how to get the transcript reliably. Enter Fathom.ai. This glorious little bot joins your Google Meet, transcribes, and even provides a decent AI summary right after the meeting. Crucially, it has direct Zapier integration, or at least a robust webhook capability that gives you the full transcript and summary almost immediately.

Here's the new, battle-tested Zap:

Trigger: Fathom.ai 'New Meeting Summary' event. This gives me the full transcript and Fathom's own summary.
Action 1: OpenAI GPT-4 Turbo (Model: gpt-4-1106-preview). This is where the magic for action item extraction happens. Fathom's summary is good, but I wanted our specific action items. My prompt was meticulously crafted:

Extract all distinct action items from the following Google Meet transcript.
An action item is a task or responsibility assigned to a person, typically starting with a verb.
List them clearly, including the assignee if explicitly mentioned, and a brief description of the task.
If no assignee is mentioned, simply list the task.
Format your output as a numbered list.
If no action items are present, output 'No specific action items were identified.'

Transcript:
"""
{{zap_data_transcript}}
"""

The difference in accuracy with GPT-4 and this focused prompt compared to my initial attempts with gpt-3.5-turbo was night and day. It almost always nailed the action items, even when the context was a bit fuzzy.
Action 2: Notion 'Create Database Item'. I have a Notion database called Standup Notes.
- Name: Standup {{zap_data_date}} - {{zap_data_title}}
- Summary: {{zap_data_summary}} (Fathom's summary is generally quite good for an overview)
- Action Items (text property): {{openai_action_items}} (The output from my GPT-4 step).

This setup has been running flawlessly for weeks now. Our standup notes are automatically created, searchable, and most importantly, action items are captured reliably. No more manual note-taking, no more lost tasks. The initial struggle was real, but finding the right tools and getting surgical with the AI prompt made all the difference. Sometimes, the solution isn't more complex tech, but picking the tool that does one thing exceptionally well, and then integrating it intelligently.