loading...

Hacking PouchDB to Use It on React Native

craftzdog profile image Takuya Matsuyama ・4 min read

cover

Hi, it's Takuya here.

I'm developing a React Native app which needs to sync data with CouchDB.
PouchDB, a well-designed database that can sync with CouchDB written in JavaScript for browsers, can be also used on React Native.
I have built react-native-sqlite-2 and pouchdb-adapter-react-native-sqlite to let it work on RN.

But there are still some issues to make it work.
For example, it has a big problem with storing attachments since RN doesn't support FileReader.readAsArrayBuffer yet (I hope they will support in the future though).

It seems like pouchdb-react-native project is inactive these days.
So I've been working on making PouchDB possible to run perfectly on RN and successfully did it. Here is what I did.

How to run PouchDB on RN

First, I would like to show you how to use PouchDB on RN.
I've made some packages that I hacked based on the official PouchDB core modules.
Here is a working demo app.
I am planning to use it in my production app called Inkdrop.

Install deps

Install PouchDB core packages:

npm i pouchdb-adapter-http pouchdb-mapreduce

And install hacked packages for React Native:

npm i @craftzdog/pouchdb-core-react-native @craftzdog/pouchdb-replication-react-native 

Next, install SQLite3 engine modules:

npm i pouchdb-adapter-react-native-sqlite react-native-sqlite-2
react-native link react-native-sqlite-2

Then, install some packages to polyfill functions that PouchDB needs:

npm i base-64 events

Create polyfills

Make a js file to polyfill some functions that PouchDB needs:

import {decode, encode} from 'base-64'

if (!global.btoa) {
    global.btoa = encode;
}

if (!global.atob) {
    global.atob = decode;
}

// Avoid using node dependent modules
process.browser = true

Import it at the first line of your index.js.

Load PouchDB

Make pouchdb.js like so:

import PouchDB from '@craftzdog/pouchdb-core-react-native'
import HttpPouch from 'pouchdb-adapter-http'
import replication from '@craftzdog/pouchdb-replication-react-native'
import mapreduce from 'pouchdb-mapreduce'

import SQLite from 'react-native-sqlite-2'
import SQLiteAdapterFactory from 'pouchdb-adapter-react-native-sqlite'

const SQLiteAdapter = SQLiteAdapterFactory(SQLite)

export default PouchDB
  .plugin(HttpPouch)
  .plugin(replication)
  .plugin(mapreduce)
  .plugin(SQLiteAdapter)

If you need other plugins like pouchdb-find, just add them to it.

Use PouchDB

Then, use it as usual:

import PouchDB from './pouchdb'

function loadDB () {
  return new PouchDB('mydb.db', { adapter: 'react-native-sqlite' })
}

How I hacked PouchDB

To get it to work on React Native, we need to avoid calling FileReader.readAsArrayBuffer from PouchDB core modules.
That means we always process attachments in Base64 instead of Blob.
It can be done with a few lines of code hacks.

Where readAsArrayBuffer is called

PouchDB tries to calclulate MD5 digest for every document, which needs to call readAsArrayBuffer.

In pouchdb-binary-utils/lib/index-browser.js:

72 function readAsBinaryString(blob, callback) {
73   if (typeof FileReader === 'undefined') {
74     // fix for Firefox in a web worker
75     // https://bugzilla.mozilla.org/show_bug.cgi?id=901097
76     return callback(arrayBufferToBinaryString(
77       new FileReaderSync().readAsArrayBuffer(blob)));
78   }
79
80   var reader = new FileReader();
81   var hasBinaryString = typeof reader.readAsBinaryString === 'function';
82   reader.onloadend = function (e) {
83     var result = e.target.result || '';
84     if (hasBinaryString) {
85       return callback(result);
86     }
87     callback(arrayBufferToBinaryString(result));
88   };
89   if (hasBinaryString) {
90     reader.readAsBinaryString(blob);
91   } else {
92     reader.readAsArrayBuffer(blob);
93   }
94 }

This function is called from pouchdb-md5/lib/index-browser.js:

24 function appendBlob(buffer, blob, start, end, callback) {
25   if (start > 0 || end < blob.size) {
26     // only slice blob if we really need to
27     blob = sliceBlob(blob, start, end);
28   }
29   pouchdbBinaryUtils.readAsArrayBuffer(blob, function (arrayBuffer) {
30     buffer.append(arrayBuffer);
31     callback();
32   });
33 }

Well, how to avoid that?

Storing Attachments

Disable binary option of getAttachment method in pouchdb-core/src/adapter.js like so:

714     if (res.doc._attachments && res.doc._attachments[attachmentId]
715       opts.ctx = res.ctx;
716       // force it to read attachments in base64
717       opts.binary = false;
718       self._getAttachment(docId, attachmentId,
719                           res.doc._attachments[attachmentId], opts, callback);
720     } else {

With this change, you will always get attachments encoded in base64.

Pull Replication

We have to convert blob to base64 when fetching attachments from remote database in pouchdb-replication/lib/index.js like so:

function getDocAttachmentsFromTargetOrSource(target, src, doc) {
  var doCheckForLocalAttachments = pouchdbUtils.isRemote(src) && !pouchdbUtils.isRemote(target);
  var filenames = Object.keys(doc._attachments);

  function convertBlobToBase64(attachments) {
    return Promise.all(attachments.map(function (blob) {
      if (typeof blob === 'string') {
        return blob
      } else {
        return new Promise(function (resolve, reject) {
          var reader = new FileReader();
          reader.readAsDataURL(blob);
          reader.onloadend = function() {
            const uri = reader.result;
            const pos = uri.indexOf(',')
            const base64 = uri.substr(pos + 1)
            resolve(base64)
          }
        });
      }
    }));
  }

  if (!doCheckForLocalAttachments) {
    return getDocAttachments(src, doc)
      .then(convertBlobToBase64);
  }

  return target.get(doc._id).then(function (localDoc) {
    return Promise.all(filenames.map(function (filename) {
      if (fileHasChanged(localDoc, doc, filename)) {
        return src.getAttachment(doc._id, filename);
      }

      return target.getAttachment(localDoc._id, filename);
    }))
      .then(convertBlobToBase64);
  }).catch(function (error) {
    /* istanbul ignore if */
    if (error.status !== 404) {
      throw error;
    }

    return getDocAttachments(src, doc)
      .then(convertBlobToBase64);
  });
}

That worked!
And that's why I made both @craftzdog/pouchdb-core-react-native and @craftzdog/pouchdb-replication-react-native.
If you found any problem on them, pull requests would be welcomed here.

Posted on by:

craftzdog profile

Takuya Matsuyama

@craftzdog

Indie developer based in Osaka, Japan. A solo dev of Inkdrop: https://inkdrop.app/

Discussion

markdown guide
 

Hi takuya,

I apply your changes but still I cannot see the base64 image. I can see the _attachment field but with a weird format (maybe blob)

  • I also set the encoding to plain/text
  • In couchdb server I can see the image

But when the synchronization tries to download the image back to the device its in a wrong format.

I can see the attachment has this form: 77+977+977+....9AAAA....

Could you please advise how did you solve this?

Thanks a lot in advance

 

I have the same problem, image attachment has a weird format with 77+9UE5HDQoaCgAAAAAAAAAA...
I think it still a blob type since the structure of this object is like this:
Alt text of image

 

Looks good. I wonder if it's possible to polyfill the buffer logic rather than forking pouchdb to sort the attachment issue. Otherwise, might be worth putting together a PR for a new option to the PouchDB repo for forcing base64 when buffers are not supported/available.

 

I've tried rn-nodeify to polyfill Buffer but it didn't solve the issue.

 

Thanks so much for putting this together! I tried out the demo and it worked great. I was surprised to see that you have a simple "hello, world!" text encoded as base64. Why did you do that? Do all attachments have to be base64-encoded with your method?

 

That's because I wanted to demonstrate storing attachments works fine on RN apps. "Hello, world" text is useful for that.
Right. All attachments must be encoded in base64 to let PouchDB avoid calling readAsBinaryString.

 

PouchDB works perfectly on react native apps with 'pouchdb-react-native'.
github.com/stockulus/pouchdb-react...

 

Thanks, wanted to mention the same. You're right, except there are some issues with attachments (see github.com/stockulus/pouchdb-react...), but they are working on it.

 

Sorry to say, but it doesn't and they dont, unfortunately. Project seems to be abandoned, look at open issues.
Sure you can still use it if you don't need replication (what?), but asyncstorage would be easier.

Replication does work, but not attachment.

 

Where are these SQLite DBs located on the phone? Does it exist in the app installation directory and will get removed upon app un-installation? If so, is there any DB that I can store outside the scope of the app installation so that it doesn't get removed after un-installation?

Asking because building a very offline first accessible app where the user might not be aware enough to know that uninstalling the app might lose all offline data.

 

Hi! would it be possible to use this in order to have a preloaded database (including indexes) in a react-native app? thanks!

 
 

こんにちは Takuyaさん,

Thank you for your great work!
In the section "polyfill functions that PouchDB needs" why is "events" required to be installed?

 

I am getting this error
TypeError: WebSqlPouchCore.call is not a function