Database replication is the process of copying data from a database in one computer or server to a database in another. It is very common if we want to scale our database to increase the availability to access the data. The result is a distributed database in which users can quickly access data relevant to their tasks.
The most common and classic method of database replication is Single-leader based data replication aka Master-slave Replication - one server that receives writes from clients, and replicas draw data from there. It provides read scalability, reduced latency and better availability as compared to no replication. However, the write of data is not scalable as read. We can only write data to the master node, and read from the slave node. This flaw paves way for multi-leader and leaderless replication.
Multi-leader based data replication aka Master-master replication, same as single-leader replication, the only different is it has more than one node to accept writes.
Today I would like to share about how easy to setup Master-master replication in CouchDB. One of CouchDB's strengths is the ability to synchronize two copies of the same database. We can trigger a replication process by sending a JSON object either to the /_replicate api endpoint or storing it as a document into the _replicator database.
Let say we have a database called cars in Server A and we would like to to replicate the cars database to Server B.
POST /_replicate
{
"create_target": false,
"continuous": false,
"source": {
"url": "http://SERVER_A_HOSTNAME:5984/cars",
"headers": {
"Authorization": "YOUR_SERVER_A_CREDENTIAL"
}
},
"target": {
"url": "http://SERVER_B_HOSTNAME:5984/cars",
"headers": {
"Authorization": "YOUR_SERVER_B_CREDENTIAL"
}
}
}
I personally recommend document based-replication, by storing document to trigger the replication process as you can review back the replication history and you can stop a running replication process by deleting the document, or by updating it with its
cancelproperty set totrue.
For specifying username and password, if you are using CouchDB v3.2.* , there are 3 ways to specify your username and password:
- In an
"auth": { "basic": "..." }object: ```
{
"target": {
"url": "http://someurl.com/mydb",
"auth": {
"basic": {
"username": "$username",
"password": "$password"
}
}
},
...
}
2. In the userinfo part of the endpoint URL:
{
"target": "http://user:pass@localhost:5984/bar"
...
}
3. In an "Authorization: Basic $b64encoded_username_and_password" header:
{
"target": {
"url": "http://someurl.com/mydb",
"headers": {
"Authorization": "Basic dXNlcjpwYXNz"
}
},
...
}
If you are using the CouchDB earlier version < v3.2, you can only use the second and third method to specify the username and password. Second method is not preferred as it prevents using characters like `@` and `:` in usernames or passwords.
After we triggered a replication process, we can inspect CouchDB replication status through the active tasks API `/_active_tasks` and `/_scheduler/jobs`. For document based-replications, `/_scheduler/docs` can be used to get a complete state summary. This API is preferred as it will show the state of the replication document before it becomes a replication job.
If the database doesn't exist in the target CouchDB server, we can set the `create_target` property to `true`, it will helps us to create a new database and start replicate the data from the source database. For `Master-master replication` setup, we would like to setup a long running replication instead of one time off, just set the `continuous` property to `true`.
POST /_replicator
{
"create_target": false,
"continuous": true,
"source": {
"url": "http://SERVER_A_HOSTNAME:5984/cars",
"headers": {
"Authorization": "YOUR_SERVER_A_CREDENTIAL"
}
},
"target": {
"url": "http://SERVER_B_HOSTNAME:5984/cars",
"headers": {
"Authorization": "YOUR_SERVER_B_CREDENTIAL"
}
}
}
Once we has triggered a `continuous database replication` from `Server A` to `Server B`, then we can trigger one more `continuous database replication` from `Server B` to `Server A` by switching the source and target url.
POST /_replicator
{
"create_target": false,
"continuous": true,
"source": {
"url": "http://SERVER_B_HOSTNAME:5984/cars",
"headers": {
"Authorization": "YOUR_SERVER_A_CREDENTIAL"
}
},
"target": {
"url": "http://SERVER_A_HOSTNAME:5984/cars",
"headers": {
"Authorization": "YOUR_SERVER_B_CREDENTIAL"
}
}
}
🎉That's it, now we have both `cars` databases in `Server A` and `Server B` in sync. We can even control which documents to be replicate by using [Selector Objects](https://docs.couchdb.org/en/stable/replication/replicator.html#selectorobj) or [Filter Functions](https://docs.couchdb.org/en/stable/ddocs/ddocs.html#filterfun).
For more detail about CouchDB Replication, please check out the [official documentation](https://docs.couchdb.org/en/stable/replication/intro.html). Thank you for reading.
Top comments (3)
Hey thanks for sharing, just getting started with CouchDB 🙏
Q)
do I understand it right that
I can set replication only between 2 db
so I have to set replication on every db
means
# A -- a <-> b
# B -- b <-> c
# C -- c <-> a "
?
Yes correct. Depending on how many replication you want to set. Above I used 3 replica as an example, however if your situation has more, u can replicate to more as long as you defined the replication document from which source to which target DB.