Overview of My Submission

I loaded over 300,000 Public Venmo Transactions into a Mongo Atlas database and use Atlas' Full Text Search to search by transaction memo.
Venmo transactions are interesting for full text search because users have to leave a public memo on every transaction. What relationships can you infer between Venmo users, based on their transaction memo? Are they friends, roommates, family, coworkers, lovers, or something else?

Deployed Application:

https://reidjs.github.io/venmo-search-frontend/

Submission Category:

Choose Your Own Adventure

Link to Code

Frontend Repo:

reidjs / venmo-search-frontend

Venmo Search

Deployed here:

https://reidjs.github.io/venmo-search-frontend/

About

Uses MongoDB's Atlas Search to search hundreds of thousands of Venmo transactions. Made for dev.to's mongodb hackathon

Why

I built this to explore using MongoDB's Atlas Search as an alternative to ElasticSearch for full text search. I used Venmo data because

1. it's public
2. users are forced to leave a text "comment" for every transaction

Usage

Production

1. Go to https://reidjs.github.io/venmo-search-frontend/
2. Type a query

Local Development

1. Start backend https://github.com/reidjs/venmo-search-server
2. Start frontend with npm run dev
3. Open browser to localhost:8000

Notes

Venmo dataset from https://github.com/sa7mon/venmo-data

• Out of the 7,000,000 transactions, I only used about 342,000 venmo transactions (free tier Mongo Atlas)
• To decide which transactions went into the database, I 'ranked' each transaction according to how long the comment was. Longer comment = more likely to be in database.

How to load in BSON data to atlas: mongorestore --uri mongodb+srv://[USERNAME]:[PASSWORD]@cluster0.abbj1.mongodb.net/[DATABASE] venmo.bson

How to Load in JSON data…

View on GitHub

Additional Resources / Info

Backend Repo:

reidjs / venmo-search-server

https://dashboard.heroku.com/apps/venmo-search

https://venmo-search.herokuapp.com/?q=hello

deploy changes to heroku: npm run deploy

Run locally: npm start

Looped through 7 million records to calculate the score on each one This nearly set my 2.6GHZ 6Core i7 32GB MBPro on fire In hindsight maybe I should have calculated a virtual field w/ mongoose, but this was the easy, dumb way of doing it. After about an hour I had at least 500,000 scores calculated, which was a big enough dataset to load into atlas

View on GitHub

Comments

[Padmashree Jha]

it is not working

[reid j sherman]

Thanks for letting me know! Should be fixed now.

[Padmashree Jha]

Yep