Venmo Search

reid j sherman — Mon, 03 Jan 2022 22:36:24 +0000

Overview of My Submission

I loaded over 300,000 Public Venmo Transactions into a Mongo Atlas database and use Atlas' Full Text Search to search by transaction memo.
Venmo transactions are interesting for full text search because users have to leave a public memo on every transaction. What relationships can you infer between Venmo users, based on their transaction memo? Are they friends, roommates, family, coworkers, lovers, or something else?

Deployed Application:

https://reidjs.github.io/venmo-search-frontend/

Submission Category:

Choose Your Own Adventure

Link to Code

Frontend Repo:

reidjs / venmo-search-frontend

Venmo Search

Deployed here:

https://reidjs.github.io/venmo-search-frontend/

About

Uses MongoDB's Atlas Search to search hundreds of thousands of Venmo transactions. Made for dev.to's mongodb hackathon

Why

I built this to explore using MongoDB's Atlas Search as an alternative to ElasticSearch for full text search. I used Venmo data because

it's public
users are forced to leave a text "comment" for every transaction

Usage

Production

Go to https://reidjs.github.io/venmo-search-frontend/
Type a query

Local Development

Start backend https://github.com/reidjs/venmo-search-server
Start frontend with npm run dev
Open browser to localhost:8000

Notes

Venmo dataset from https://github.com/sa7mon/venmo-data

Out of the 7,000,000 transactions, I only used about 342,000 venmo transactions (free tier Mongo Atlas)
To decide which transactions went into the database, I 'ranked' each transaction according to how long the comment was. Longer comment = more likely to be in database.

How to load in BSON data to atlas: mongorestore --uri mongodb+srv://[USERNAME]:[PASSWORD]@cluster0.abbj1.mongodb.net/[DATABASE] venmo.bson

How to Load in JSON data…

View on GitHub

Additional Resources / Info

Backend Repo:

reidjs / venmo-search-server

https://dashboard.heroku.com/apps/venmo-search

https://venmo-search.herokuapp.com/?q=hello

deploy changes to heroku: npm run deploy

Run locally: npm start

Looped through 7 million records to calculate the score on each one This nearly set my 2.6GHZ 6Core i7 32GB MBPro on fire In hindsight maybe I should have calculated a virtual field w/ mongoose, but this was the easy, dumb way of doing it. After about an hour I had at least 500,000 scores calculated, which was a big enough dataset to load into atlas

View on GitHub

DEV Community: reid j sherman

Venmo Search

Overview of My Submission

Submission Category:

Link to Code

reidjs / venmo-search-frontend

Venmo Search

About

Why

Usage

Notes

Additional Resources / Info

reidjs / venmo-search-server