Vibe Coding a Receipt Parser in an Hour (with VLMs + Google Sheets)

Today I Tried Out Vibe Coding For Real.

Without too much manual review or prep. I wanted to stop resisting and depend on GENAI for 96% of the dev work. It wasn’t perfect but it was fast, lean, and surprisingly powerful. I finished an mvp within 60mins give or take

The Idea

I’ve been thinking a lot about personal efficiency — the kind that starts at home.

What if I could keep track of the meals I cook, the ingredients I buy, and the patterns that keep showing up?

Not with an app from the Play Store, but with something I designed, shaped by how I actually live.

Because I think that’s where GenAI really shines:
Not just answering questions, but help you build mental systems for repetitive tasks around your life. Its like a Siri version of Jarvis ?

I came back from the store with a receipt in hand and thought:
"Why enter all this manually when I could just take a picture and let an agent do the work?"

So the raw idea was simple:

• Upload an image of my receipt
• Extract ingredients and prices
• Connect the agent to a spreadsheet to update repetitive information
• Update info like shelf life (how long they lasted), what meal i made, later manually

(Note: this wasn’t the sheet. — I changed the order of elements later on)

Chatgpt: Chief Architect

I made Chatgpt the chief architect. It created a nice read me of the plan filled with mini objectives and then i used that Readme as kinda the prompt for copilot in vscode.
The readMe had the skeleton, gauged to guide copilot. Cause we know what happens when youre not specific with what you want. Theyll go off the leash.

meal-prepping/
├── app/
│   ├── main.py              # FastAPI app: upload, process, submit routes
│   ├── agent/
│   │   ├── ocr.py           # OCR wrapper to extract text from images
│   │   ├── parser.py        # Parse OCR text into structured data
│   │   └── infer.py         # Clean up quantities and prices
│   ├── services/
│   │   └── sheets.py        # Google Sheets integration
│   ├── templates/
│   │   └── review.html      # Review page with editable table
│   └── static/
│       └── styles.css       # Basic styling
├── tests/
│   └── test_parser.py       # Unit tests
├── requirements.txt         # Dependencies
└── README.md                <--- This is where you are   

App Flow (as defined by the agent)

[1] Upload receipt image
        ↓
[2] OCR extracts text
        ↓
[3] Parse into structured data
        ↓
[4] Review/edit in table
        ↓
[5] Send to Google Sheets

📦 Dependencies (Before Openai and dotenv)

fastapi
uvicorn
pillow
easyocr
google-api-python-client
google-auth
python-multipart

Copilot: The builder

We decided on the requirements and got to work. Copilot had no issues. Its first review was on the name of the project and how it conflicted with the information in the Readme file. That was nothing major. Aside that i had no bugs for my first prototype

OCR Results (First Attempt)

This was the image i consistently used for testing

After our first test. I noticed that easyocr wasnt giving us good results.

(It got the prices right but couldnt extract the ingredients)

Switching to VLMs Changed Everything

So i decided to go fully into the direction of VLMS. And it did way better. More than expected.

It was able to get all the ingredients right. But about 70% of the prices right. I couldn't blame it, because to be fair the paper was crumbled. Which is actually grea. since i have the ability to correct it before we submit it to our spreadsheet.

Submitting to spreadsheet.

The only issue i had here was that the information being sent did not match the columns. So i cheated a bit. After sending the information i change the columns appropriately

PS:
To get the full system working end-to-end — from receipt image to structured spreadsheet — I had to wire up a few things behind the scenes.

Here’s exactly how I did it, so you don’t get stuck:

1. Google Sheets API Setup

You can’t just connect to Google Sheets without letting Google know your app exists.

Here’s the flow:

• Go to the Google Cloud Console.

• Create a new project (e.g. MealPrepAgent).

• Go to APIs & Services → Library and search for:
  👉 Google Sheets API

• Click Enable to turn it on for your project.

Boom, your project is now allowed to talk to spreadsheets.

2. Get Your Spreadsheet ID

This part is super simple but easy to miss.

Open the Google Sheet you want to connect to. Look at the URL:

bash

https://docs.google.com/spreadsheets/d/<some random figure and text >/edit#gid=0

Copy just the part between /d/ and /edit.

That’s your Spreadsheet ID. Save it in your .env like this:

SPREADSHEET_ID=<some random figure and text >

3. Set Up Your Service Account & Credentials

This is what lets your Python app act like a Google user (with permissions).

Here’s how:

• Go back to the Google Cloud Console.

• Go to IAM & Admin → Service Accounts.

• Click “Create Service Account”

  • Name: receipt-agent or whatever
  • Permissions: just click through — you don’t need to set roles here

• After creating, go to the Keys tab → Add Key → Create new key; Choose JSON

• Download it — this is your credentials.json

• Put this file in your project root. Then, in your .env:
  
  

GOOGLE_CREDENTIALS_PATH=credentials.json