loading...
Cover image for Saving Data in JavaScript Without a Database

Saving Data in JavaScript Without a Database

healeycodes profile image Andrew Healey Updated on ・5 min read

You've just written a great piece of JavaScript. But when the running process stops, or the user refreshes, all of that nice data disappears into the ether.

Is this you?

When prototyping, or otherwise working on tiny projects, it can be helpful to manage some state without resorting to a database solution that wasn't designed for that creative itch you're trying to scratch.

We're going to explore some options that I wish I knew about when I started tinkering on the web. We'll look at JavaScript in the browser and Node.js on the back end. We'll also look at some lightweight databases that use the local file system.

Node.js

First up is JSON serializing your data and saving it to disk. The MDN Docs have a great article if you haven't worked with JSON before.

const fs = require('fs');

const users = {
    'Bob': {
        age: 25,
        language: 'Python'
    },
    'Alice': {
        age: 36,
        language: 'Haskell'
    }
}

fs.writeFile('users.json', JSON.stringify(users), (err) => {  
    // Catch this!
    if (err) throw err;

    console.log('Users saved!');
});

We created our users object, converted it to JSON with JSON#stringify and called fs#writeFile. We passed in a filename, our serialized data, and an arrow function as a callback to execute when the write operation finishes. Your program will continue executing code in the meanwhile.

You can also use this method to write normal serialized data by passing in anything that can be cast to a string. If you're storing text data, you may find fs#appendFile useful. It uses an almost identical API but sends the data to the end of the file, keeping the existing contents.

There is a synchronous option, fs#writeFileSync but it isn't recommended as your program will be unresponsive until the write operation finishes. In JavaScript, you should aim to Never Block.

If you're dealing with CSV files, reach for the battle-hardened node-csv project.

Let's load those users back into our program with fs#readFile.

fs.readFile('user.json', (err, data) => {
    // Catch this!
    if (err) throw err;

    const loadedUsers = JSON.parse(data);
    console.log(loadedUsers);
});

Lightweight databases

SQLite uses a local file as a database β€” and is one of my favorite pieces of software in the world. It enables many of my smaller projects to exist with low maintenance and little deploying hassle.

Here are some facts about SQLite:

  • The project has 711 times as much test code and test scripts compared to other code.
  • The developers pledge to keep it backward compatible through at least the year 2050.
  • It's used on planes, in Android, and you probably interacted with it in some way on your way to this article today.

Seriously, How SQLite Is Tested is a wild ride.

In Node.js, we commonly use the sqlite3 npm package. I'll be using some code from Glitch's hello-sqlite template, which you can play around with and remix without an account.

// hello-sqlite
var fs = require('fs');
var dbFile = './.data/sqlite.db'; // Our database file
var exists = fs.existsSync(dbFile); // Sync is okay since we're booting up
var sqlite3 = require('sqlite3').verbose(); // For long stack traces
var db = new sqlite3.Database(dbFile);

Through this db object, we can interact with our local database like we would through a connection to an outside database.

We can create tables.

db.run('CREATE TABLE Dreams (dream TEXT)');

Insert data (with error handling).

db.run('INSERT INTO Dreams (dream) VALUES (?)', ['Well tested code'], function(err) {
  if (err) {
    console.error(err);
  } else {
    console.log('Dream saved!');
    }
});

Select that data back.

db.all('SELECT * from Dreams', function(err, rows) {
  console.log(JSON.stringify(rows));
});

You may want to consider serializing some of your database queries. Each command inside the serialize() function is guaranteed to finish executing before the next one starts. The sqlite3 documentation is expansive. Keep an eye on the SQLite Data Types as they can be a little different to other databases.

If even SQLite seems like too much overhead for your project, consider lowdb (also remixable on Glitch). lowdb is exciting because it's a small local JSON database powered by Lodash (supports Node, Electron and the browser). Not only does it work as a wrapper for JSON files on the back end it also provides an API which wraps localStorage in the browser.

From their examples:

import low from 'lowdb'
import LocalStorage from 'lowdb/adapters/LocalStorage'

const adapter = new LocalStorage('db')
const db = low(adapter)

db.defaults({ posts: [] })
  .write()

// Data is automatically saved to localStorage
db.get('posts')
  .push({ title: 'lowdb' })
  .write()

Browser

This brings us to the front end. window#localStorage is the modern solution to storing data in HTTP cookies β€” which MDN doesn't recommend for storing things anymore.

Let's interact with them right now. If you're on desktop, open your dev console (F12 on Chrome) and see what DEV is storing for you:

for (const thing in localStorage) {
  console.log(thing, localStorage.getItem(thing))
}

// Example of one thing:
// pusherTransportTLS {"timestamp":1559581571665,"transport":"ws","latency":543}

We saw how lowdb interacted with localStorage but for our small projects it's probably easier to talk to the API directly. Like this:

// As a script, or in console
localStorage.setItem('Author', 'Andrew') // returns undefined
localStorage.getItem('Author') // returns "Andrew"
localStorage.getItem('Unset key') // returns null

It gets easier still: you can treat it like an object. Although, MDN recommends the API over this shortcut.

console.log(localStorage['Author']); // prints "Andrew"

If you don't want to store data on the user's computer forever (which can be cleared with localStorage.clear() but don't run this on DEV) you may be interested in sessionStorage which has a near identical API and only stores data while the user is on the page.

End notes

I read somewhere that SQLite is used onboard the Internation Space Station in some capacity but I haven't been able to find a source. My fiancΓ©e wants you to know that SQLite is a database and the title of this post is incorrect.


Join 150+ people signed up to my newsletter on programming and personal growth!

I tweet about tech @healeycodes.

Posted on by:

healeycodes profile

Andrew Healey

@healeycodes

πŸ“š 🌡 πŸ• Love blogging, open-source, and helping out. https://healeycodes.com

Discussion

markdown guide
 

Another way is (or will be) kv storage. It's the first built in module for the web.

below code copied from article:

import {storage} from 'std:kv-storage';

const main = async () => {
  const oldPreferences = await storage.get('preferences');

  document.querySelector('form').addEventListener('submit', async () => {
    const newPreferences = Object.assign({}, oldPreferences, {
      // Updated preferences go here...
    });

    await storage.set('preferences', newPreferences);
  });
};

It uses indexedDb under the hood, but provides a convenient API similar to localStorage. With the main difference that it is asynchronous and therefore doesn't block the main thread.

 

Big fan of lowdb with file sync adapter

// Database.
const path = require('path');
const low = require('lowdb');
const FileSync = require('lowdb/adapters/FileSync');
const adapter = new FileSync(path.resolve(`${__dirname}/database.json`));
const db = low(adapter);

module.exports = db;
 

So I pretty much exclusively use SQLite, just wondering when you would use a different option? Is it all personal preference, depends on the scope of the project, ease of use...? How do you decide what you're going to use when you start a (say small) project?

 

If you plan on having more than one process needing to make database calls or if you're expecting periods of very high traffic*.

SQLite works great as the database engine for most low to medium traffic websites (which is to say, most websites). The amount of web traffic that SQLite can handle depends on how heavily the website uses its database. Generally speaking, any site that gets fewer than 100K hits/day should work fine with SQLite. The 100K hits/day figure is a conservative estimate, not a hard upper bound. SQLite has been demonstrated to work with 10 times that amount of traffic.

More info here.

 

So I pretty much exclusively use SQLite, just wondering when you would use a different option? Is it all personal preference, depends on the scope of the project, ease of use...? How do you decide what you're going to use when you start a (say small) project?

Unless I actually need some functionality that's provided by having an SQL interface, I try to avoid it. I have no issue 'speaking' basic SQL, it's just that in most cases, you don't actually need it, so ti largely just adds overhead (SQL is complicated from an implementation perspective). See for example all of the major FOSS web browsers historically using SQLite for their cache, when all they really need is flat files with directory hashing and a simple 1:1 index.

Other disadvantages to SQLite specifically include it not being truly concurrent-access safe, and reimplementing much of the work the underlying filesystem is already doing.

Personally, if it's just simple data serialization, I'm a fan of YAML. It's reasonably lightweight, easily extensible, widely supported, and most importantly, can be debugged even by non-programmers. It does, of course, have it's own issues, but most of them are non-issues for my typical usage.

 

I've also used dexie in order to interact with IndexedDB in the browser, highly recommend it.

 

If you don't need more then key-value storage, idb-keyval is light and has a very similar API to localStorage.

LocalStorage should be avoided unless very specific use case (see this comment).

 

I'm not really sure of your point here, I'm recommending the use of IndexedDB not local storage.

The LocalStorage comment is for the author of the article.

 

Indexedb with pouch in browser is quite interesting as a level up from session or local storage, or there is even sqlite WebAssembly! So stored much Database.

 

I recommend something called json-server for prototyping, it's fantastic.

 

This is awesome. Thanks for the resource!

 

What about IndexedDB in the browsers for client-side storage?

developer.mozilla.org/en-US/docs/W...

But it have bad supporting in browsers like IE

 

Thank you so much for this. It's nice to know I can prototype stuff without having to figure out a database.

 

This article is right to the point.thx

 
 
 

Found small typo:
"fs.readFile('user.json'..." shuold be: "users.json".
Thanx for the great article!

 

Fixed! Thanks for the good spot πŸ‘

 

Lovely!

I recently deployed a system for a client with SQLite, and it thruly is a great DB system even in production environment. 100% recommended.