DEV Community

Takuya Matsuyama
Takuya Matsuyama

Posted on • Updated on

Hacking PouchDB to Use It on React Native

cover

UPDATE(2021/02/12)


Hi, it's Takuya here.

I'm developing a React Native app which needs to sync data with CouchDB.
PouchDB, a well-designed database that can sync with CouchDB written in JavaScript for browsers, can be also used on React Native.
I have built react-native-sqlite-2 and pouchdb-adapter-react-native-sqlite to let it work on RN.

But there are still some issues to make it work.
For example, it has a big problem with storing attachments since RN doesn't support FileReader.readAsArrayBuffer yet (I hope they will support in the future though).

It seems like pouchdb-react-native project is inactive these days.
So I've been working on making PouchDB possible to run perfectly on RN and successfully did it. Here is what I did.

How to run PouchDB on RN

First, I would like to show you how to use PouchDB on RN.
I've made some packages that I hacked based on the official PouchDB core modules.
Here is a working demo app.
I am planning to use it in my production app called Inkdrop.

Install deps

Install PouchDB core packages:

npm i pouchdb-adapter-http pouchdb-mapreduce
Enter fullscreen mode Exit fullscreen mode

And install hacked packages for React Native:

npm i @craftzdog/pouchdb-core-react-native @craftzdog/pouchdb-replication-react-native 
Enter fullscreen mode Exit fullscreen mode

Next, install SQLite3 engine modules:

npm i pouchdb-adapter-react-native-sqlite react-native-sqlite-2
react-native link react-native-sqlite-2
Enter fullscreen mode Exit fullscreen mode

Then, install some packages to polyfill functions that PouchDB needs:

npm i base-64 events
Enter fullscreen mode Exit fullscreen mode

Create polyfills

Make a js file to polyfill some functions that PouchDB needs:

import {decode, encode} from 'base-64'

if (!global.btoa) {
    global.btoa = encode;
}

if (!global.atob) {
    global.atob = decode;
}

// Avoid using node dependent modules
process.browser = true
Enter fullscreen mode Exit fullscreen mode

Import it at the first line of your index.js.

Load PouchDB

Make pouchdb.js like so:

import PouchDB from '@craftzdog/pouchdb-core-react-native'
import HttpPouch from 'pouchdb-adapter-http'
import replication from '@craftzdog/pouchdb-replication-react-native'
import mapreduce from 'pouchdb-mapreduce'

import SQLite from 'react-native-sqlite-2'
import SQLiteAdapterFactory from 'pouchdb-adapter-react-native-sqlite'

const SQLiteAdapter = SQLiteAdapterFactory(SQLite)

export default PouchDB
  .plugin(HttpPouch)
  .plugin(replication)
  .plugin(mapreduce)
  .plugin(SQLiteAdapter)
Enter fullscreen mode Exit fullscreen mode

If you need other plugins like pouchdb-find, just add them to it.

Use PouchDB

Then, use it as usual:

import PouchDB from './pouchdb'

function loadDB () {
  return new PouchDB('mydb.db', { adapter: 'react-native-sqlite' })
}
Enter fullscreen mode Exit fullscreen mode

How I hacked PouchDB

To get it to work on React Native, we need to avoid calling FileReader.readAsArrayBuffer from PouchDB core modules.
That means we always process attachments in Base64 instead of Blob.
It can be done with a few lines of code hacks.

Where readAsArrayBuffer is called

PouchDB tries to calclulate MD5 digest for every document, which needs to call readAsArrayBuffer.

In pouchdb-binary-utils/lib/index-browser.js:

72 function readAsBinaryString(blob, callback) {
73   if (typeof FileReader === 'undefined') {
74     // fix for Firefox in a web worker
75     // https://bugzilla.mozilla.org/show_bug.cgi?id=901097
76     return callback(arrayBufferToBinaryString(
77       new FileReaderSync().readAsArrayBuffer(blob)));
78   }
79
80   var reader = new FileReader();
81   var hasBinaryString = typeof reader.readAsBinaryString === 'function';
82   reader.onloadend = function (e) {
83     var result = e.target.result || '';
84     if (hasBinaryString) {
85       return callback(result);
86     }
87     callback(arrayBufferToBinaryString(result));
88   };
89   if (hasBinaryString) {
90     reader.readAsBinaryString(blob);
91   } else {
92     reader.readAsArrayBuffer(blob);
93   }
94 }
Enter fullscreen mode Exit fullscreen mode

This function is called from pouchdb-md5/lib/index-browser.js:

24 function appendBlob(buffer, blob, start, end, callback) {
25   if (start > 0 || end < blob.size) {
26     // only slice blob if we really need to
27     blob = sliceBlob(blob, start, end);
28   }
29   pouchdbBinaryUtils.readAsArrayBuffer(blob, function (arrayBuffer) {
30     buffer.append(arrayBuffer);
31     callback();
32   });
33 }
Enter fullscreen mode Exit fullscreen mode

Well, how to avoid that?

Storing Attachments

Disable binary option of getAttachment method in pouchdb-core/src/adapter.js like so:

714     if (res.doc._attachments && res.doc._attachments[attachmentId]
715       opts.ctx = res.ctx;
716       // force it to read attachments in base64
717       opts.binary = false;
718       self._getAttachment(docId, attachmentId,
719                           res.doc._attachments[attachmentId], opts, callback);
720     } else {
Enter fullscreen mode Exit fullscreen mode

With this change, you will always get attachments encoded in base64.

Pull Replication

We have to convert blob to base64 when fetching attachments from remote database in pouchdb-replication/lib/index.js like so:

function getDocAttachmentsFromTargetOrSource(target, src, doc) {
  var doCheckForLocalAttachments = pouchdbUtils.isRemote(src) && !pouchdbUtils.isRemote(target);
  var filenames = Object.keys(doc._attachments);

  function convertBlobToBase64(attachments) {
    return Promise.all(attachments.map(function (blob) {
      if (typeof blob === 'string') {
        return blob
      } else {
        return new Promise(function (resolve, reject) {
          var reader = new FileReader();
          reader.readAsDataURL(blob);
          reader.onloadend = function() {
            const uri = reader.result;
            const pos = uri.indexOf(',')
            const base64 = uri.substr(pos + 1)
            resolve(base64)
          }
        });
      }
    }));
  }

  if (!doCheckForLocalAttachments) {
    return getDocAttachments(src, doc)
      .then(convertBlobToBase64);
  }

  return target.get(doc._id).then(function (localDoc) {
    return Promise.all(filenames.map(function (filename) {
      if (fileHasChanged(localDoc, doc, filename)) {
        return src.getAttachment(doc._id, filename);
      }

      return target.getAttachment(localDoc._id, filename);
    }))
      .then(convertBlobToBase64);
  }).catch(function (error) {
    /* istanbul ignore if */
    if (error.status !== 404) {
      throw error;
    }

    return getDocAttachments(src, doc)
      .then(convertBlobToBase64);
  });
}
Enter fullscreen mode Exit fullscreen mode

That worked!
And that's why I made both @craftzdog/pouchdb-core-react-native and @craftzdog/pouchdb-replication-react-native.
If you found any problem on them, pull requests would be welcomed here.

Top comments (19)

Collapse
 
badbod99 profile image
badbod99

Looks good. I wonder if it's possible to polyfill the buffer logic rather than forking pouchdb to sort the attachment issue. Otherwise, might be worth putting together a PR for a new option to the PouchDB repo for forcing base64 when buffers are not supported/available.

Collapse
 
craftzdog profile image
Takuya Matsuyama

I've tried rn-nodeify to polyfill Buffer but it didn't solve the issue.

Collapse
 
garfieldnate profile image
Nathan Glenn

Thanks so much for putting this together! I tried out the demo and it worked great. I was surprised to see that you have a simple "hello, world!" text encoded as base64. Why did you do that? Do all attachments have to be base64-encoded with your method?

Collapse
 
craftzdog profile image
Takuya Matsuyama

That's because I wanted to demonstrate storing attachments works fine on RN apps. "Hello, world" text is useful for that.
Right. All attachments must be encoded in base64 to let PouchDB avoid calling readAsBinaryString.

Collapse
 
juliogarma profile image
Julio Márquez • Edited

Hi takuya,

I apply your changes but still I cannot see the base64 image. I can see the _attachment field but with a weird format (maybe blob)

  • I also set the encoding to plain/text
  • In couchdb server I can see the image

But when the synchronization tries to download the image back to the device its in a wrong format.

I can see the attachment has this form: 77+977+977+....9AAAA....

Could you please advise how did you solve this?

Thanks a lot in advance

Collapse
 
duytq94 profile image
Duy Tran • Edited

I have the same problem, image attachment has a weird format with 77+9UE5HDQoaCgAAAAAAAAAA...
I think it still a blob type since the structure of this object is like this:
Alt text of image

Collapse
 
vittaljk profile image
vittal

I am getting this error
TypeError: WebSqlPouchCore.call is not a function

Collapse
 
productivix profile image
productivix

hi, how did you solve this issue please ?

Collapse
 
sl0tr profile image
Mohaimin Islam

Where are these SQLite DBs located on the phone? Does it exist in the app installation directory and will get removed upon app un-installation? If so, is there any DB that I can store outside the scope of the app installation so that it doesn't get removed after un-installation?

Asking because building a very offline first accessible app where the user might not be aware enough to know that uninstalling the app might lose all offline data.

Collapse
 
snehalagrawal profile image
SnehalAgrawal
Collapse
 
duytq94 profile image
Duy Tran

Thank you so much!

Collapse
 
productivix profile image
productivix

the example does not work : see issue

Collapse
 
cilvet profile image
Iñigo Cilveti

Hi! would it be possible to use this in order to have a preloaded database (including indexes) in a react-native app? thanks!

Collapse
 
snehalagrawal profile image
SnehalAgrawal

Yes you can, You just need to do some addition things
Example to Load Pre Populated SQLite Database in React Native

Collapse
 
rdewolff profile image
Rom

Thanks, wanted to mention the same. You're right, except there are some issues with attachments (see github.com/stockulus/pouchdb-react...), but they are working on it.

Collapse
 
greenais profile image
Alexander

Sorry to say, but it doesn't and they dont, unfortunately. Project seems to be abandoned, look at open issues.
Sure you can still use it if you don't need replication (what?), but asyncstorage would be easier.

Thread Thread
 
rdewolff profile image
Rom

Replication does work, but not attachment.

Collapse
 
hayate profile image
Andrea Belvedere

こんにちは Takuyaさん,

Thank you for your great work!
In the section "polyfill functions that PouchDB needs" why is "events" required to be installed?