DEV Community

loading...

Need Help, building off-line & file based airtable

madibalive profile image Razak Wasiu ・1 min read

Web apps like notion,coda,airtable bring new dynamics to spreadsheet database, but most of them are backed up with back-end database,primarily online first and only allow for import and export, for local copy.

What is the best direction to building similar product but limited scope(ie small feature) and most importantly off-line first using file based storage as database, like excel, since am a programmer interact with file storage using sql like abstraction but allow for large dataset.

Basically off-line excel using web technology(react + electron ) but without bundling a database for the app to work, and off-line 90% of the time, thus using file as storage, but program interact with file using a sql abstraction. in future add express server for collaboration.

thanks for the help, this is for insurance company i work at, in Africa, so off-line is crucial.

Discussion (6)

Collapse
rakannimer profile image
Rakan Nimer

Have you considered using sqlite ? Stores its data on the filesystem and is query-able and pretty solid 👍

Collapse
madibalive profile image
Razak Wasiu Author • Edited

i research on websql and sqlite, how is the performance on a 500k - 1 million rows
i do alot java ee, and performance is crucial havent work sqlite or websql with 500k + rows , is performance passable and ensure correctness ? thanks for the reply

Collapse
rakannimer profile image
Rakan Nimer

Here's a good overview of when to use sqlite : sqlite.org/whentouse.html

Querying 500K - 1M rows of well structured and indexed data won't be the problem (unless you want to do very complex queries), it's more likely that concurrent writes will be your bottleneck, but in your use-case it doesn't seem like you'll have that problem ?

Hope that helps !
Cheers

Thread Thread
madibalive profile image
Razak Wasiu Author

Hi thanks for the reply,my plan is to switch to hosted mysql for collaboration, so use sqlite for single user ,over electron basically like excel, do i face concurrency using single user,mostly multiple promise chain query?

Thread Thread
rakannimer profile image
Rakan Nimer

Using it as a mysql caching layer for single-user should work smoothly. At worst, if you face performance issues you can trim old data.

Make sure to check out better-sqlite3. It offers a sync, fast API for sqlite.

Thread Thread
madibalive profile image
Razak Wasiu Author

thanks for the all the help. off to read the docs on sqlite but be best contender

Forem Open with the Forem app