How to setup Clouseau for CouchDB search on Unix-like systems

Introduction

Clouseau is written in Scala code and provides access to the Lucene library. You can use Lucene Query Syntax to run search queries. In previous article I did talk about we can query in Apache CouchDB with Clouseau, today I would like to talk about how to setup Clouseau that link with CouchDB.

---

Prerequisites📜

Before you begin this guide, you should have a regular, non-root user with sudo privileges configured on your server. Additionally, you will need to have a up and running CouchDB version v3+ on your server. You can learn how to install Apache CouchDB on Unix-like systems by following the official documentation here. Besides that, since Clouseau is written in Scala, we also need to install Java to run Clouseau.

The search plugin is runtime-compatible with Java JDKs 6, 7 and 8. Building a release from source requires JDK 6. It will not work with any newer version of Java.

In this tutorial, I am using Ubuntu 20.04, JDK 8 and CouchDB v3.2.1 . CouchDB v3 and higher is a must as CouchDB v3 has extremely simplified the setup of Lucene search.

1. Download Clouseau

You can download the latest Clouseau release from here. Your Clouseau might be different depending on the time you download. My current latest version is Clouseau 2.17.0. I am using wget to download from my Ubuntu server.

sudo wget https://github.com/cloudant-labs/clouseau/releases/download/2.17.0/clouseau-2.17.0-dist.zip

2. Extract Clouseau zip

We are using unzip to extract the Clouseau jar files.

Tips: If you don't have unzip, you can run the command sudo apt-get install unzip for Ubuntu.

Then, we can unzip it with the command below:

sudo unzip clouseau-2.17.0-dist.zip -d /opt/

As you can see, we extract the Clouseau to the destination /opt/
. Now we can navigate to the extracted destination and view the extracted files.

cd /opt/clouseau-2.17.0/

3. Create Configuration files

According to CouchDB official documentation, in order to setup the search plugin, there are 2 configuration files required called clouseau.ini and log4j.properties with the following content:

clouseau.ini

[clouseau]

; the name of the Erlang node created by the service, leave this unchanged
name=clouseau@127.0.0.1

; set this to the same distributed Erlang cookie used by the CouchDB nodes
cookie=monster

; the path where you would like to store the search index files
dir=/opt/couchdb/share/

; the number of search indexes that can be open simultaneously
max_indexes_open=500

log4j.properties

log4j.rootLogger=debug, CONSOLE
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} %c [%p] %m%n

4. Create a script to run Clouseau

Below is the script to run Clouseau. If you have a Java background, below syntax should be familiar for you. In this case, I will place the script file under /opt/clouseau-2.17.0/bin/ and the file name called clouseau, full path would be /opt/clouseau-2.17.0/bin/clouseau

#!/bin/sh
/usr/bin/java -server \
     -Xmx2G \
     -Dsun.net.inetaddr.ttl=30 \
     -Dsun.net.inetaddr.negative.ttl=30 \
     -Dlog4j.configuration=file:/opt/clouseau-2.17.0/log4j.properties \
     -XX:OnOutOfMemoryError="kill -9 %p" \
     -XX:+UseConcMarkSweepGC \
     -XX:+CMSParallelRemarkEnabled \
     -classpath '/opt/clouseau-2.17.0/*' \
     com.cloudant.clouseau.Main \
     /opt/clouseau-2.17.0/clouseau.ini

Your path could be different, it depends on where you put the clouseau files, you could change the /opt/clouseau-2.17.0/ if needed, in this case, I'm putting all the clouseau files and configurations under /opt/clouseau-2.17.0 directories.

We are almost there. Now we want to tell Ubuntu to run the service when the server startup. To do that, we can navigate to /lib/systemd/system

cd /lib/systemd/system

and create a file called clouseau.service with the content below:

[Unit]
Description=clouseau service

[Service]
WorkingDirectory=/opt/clouseau-2.17.0
ExecStart=/opt/clouseau-2.17.0/bin/clouseau
Restart=always
RestartSec=3
User=root

[Install]
WantedBy=multi-user.target

Once created, then run the 3 commands below:

sudo systemctl daemon-reload

sudo systemctl enable clouseau.service

sudo systemctl start clouseau

Then, Clouseau should already be up and running. Check with the systemd init system to make sure the service is running by typing

sudo systemctl status apache2

Finally, we can check whether our Apache CouchDB and Clouseau is connected by using CouchDB root endpoint.

curl 127.0.0.1:5984

If Clouseau is connected to CouchDB successfully, You should see the search keyword display under features.

That's it, we did it~🎉

Now we can query with Lucene Query Syntax in CouchDB.
Hope you find this guide useful.👋

Comments

[efinley1272]

I followed this procedure (used version 2.21.1 instead of 2.17), but didn't get "search" in the "features" output of couchdb. clouseau is running as verified by systemctl status clouseau, but I don't get any logging from clouseau to troubleshoot because of:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation

I'm flailing in the dark when it comes to Java. Any pointers would be appreciated.

[Jordan Soo Yen Yih]

Hi efinley, May I know what CouchDB version you are using? And, did you create any search index? If yes, you may try to query with the search index, because recently I have experienced the same issue on one of my CouchDB server and I need to query with the search index one time then only the "search" will show in the "features". Let me know whether it works for you.

[efinley1272]

I'm using version 2.21.1. I have NOT tried creating a search index and searching with it. I will try that and report back.

[Jordan Soo Yen Yih]

oh... you can't use CouchDB v2 and lower. You must use CouchDB v3 and higher as I said in the article above

[efinley1272]

Sorry, I mean the version of Clouseau is 2.21.1 the version of CouchDB is 3.2.2.

[Jordan Soo Yen Yih]

I see. Then you may try to create a search index and query it.

[efinley1272]

Sorry for taking so long to reply. Life got crazy and I just got back to it today. I added a database 'test', then the 'search' feature showed up. I did NOT create a search index or even add any documents to the 'test' database. I did NOT check for the search feature today before adding the test database so I'm not sure whether it was adding the database or just waiting that made the feature show up.

[Jordan Soo Yen Yih]

I see. Do you mind sharing the server spec you are using? This is because from my experience, low server spec is another possibility that will cause the search feature late to show up.

[efinley1272]

This is running inside an LXD container, which is running in a Ubuntu virtual machine with 16GB RAM and 4 processors, which is running on bare metal hardware with 88 processors and 1.5TB RAM. It's the only VM on that hardware, so no contention for resources.

Interesting data point, I haven't done anything AT ALL with the database since I last posted that the 'search' feature showed up. When I queried it today, the 'search' feature is missing again. This is really depressing. Any pointers on getting logging working for Clouseau 2.21.1 would be greatly appreciated.

[efinley1272]

I'm replying to my own comment because I have some relevant data points. I was able to get logging working by putting slf4j-simple.jar in the classpath as mentioned in some of the other comments. The search feature showed up and started working after creating a search document and adding a record. I appreciate all the help from this blog and from Jordan's comments. Thank you.

I'm very much looking forward to Nouveau. Anyone know the timeline on that?

[Mies Buis]

Thanks for the tut! Hoping someone could help me, because I get

{"error":"ou_est_clouseau","reason":"Could not connect to the Clouseau Java service at clouseau@127.0.0.1"}

I am running couchdb 3.2.2 and clouseau 2.21.2. The systemctl status is "available" and I fixed a slf4j error by placing a slf4j-simple jar in the classpath. I built a simple index (as discussed here), rebooted a couple of times, but: nothing. The log in systemctl status gives me:

INFO scalang.node.ErlangHandler - channel disconnected org.jboss.blablabla
ERROR scalang.node.ServerHandshakeHandler - Channel closed during handshake

Can anybody help me find clouseau?

[Jordan Soo Yen Yih]

Hi... Seems like your CouchDB can't find Clouseau. When you curl 127.0.0.1:5984 , it didn't show the Search feature right?

[Mies Buis]

Thanks. That is correct. I do not see the search feature when I curl CouchDB, but I was not worried because of what I read in these comments. I also managed to solve the fact that Clouseau moved to another logging solution, which is not in this tutorial. (I added an slf4j-simple.jar in my Clouseau dir). The log for Clouseau shows everything copacetic untill I try to use search. Then the connection drops and times out in the browser with the "ou est clouseau" error.
I am thinking of moving to a "real" Solr installation instead. This also gives me Lucene 9 whereas Clouseau is Lucene 6 I think. I have not looked into the differences yet.
Hoping some CouchDB devs get around to actually including FTS and all nice features such as true faceting in CouchDB 4.

[Jordan Soo Yen Yih]

According to CouchDB devs latest update, there is no CouchDB 4 yet and one thing confirmed that there are no new features in CouchDB 4. But the good news is, they are currently working on some new features for CouchDB v3.3.0. One of the new features is the new Full-text search for CouchDB called Nouveau which supports Lucene 9 + DropWizard. github.com/rnewson/nouveau . If everything goes well according to the plan, then we can have the new full-text search in CouchDB v3.3.0😊

[Mies Buis]

That is excellent news! When I looked further, there was talk about moving CouchDB on top of FoundationDB, which might have been the "4" I had seen "somewhere", but apparently IBM would have funded the move and pulled the plug this march.
I will look into Nouveau and hope an official release comes soonish. Some QOL improvements in Fauxton would be nice too.

[Jordan Soo Yen Yih]

Yup, you are right, they did discuss about that before. But last week CouchDB Berlin Meetup, Janl just announced that no FoundationDB for CouchDB 4. If you ever heard about it before, then just pretend nothing happened or forget about it. 🤣

[Alvaro Cortes]

Hi Jordan, thanks for your tutorial, is by far clearer than official documentation installing, however I don't get the search working.
The clouseau.service loads right but it fails when I try to start it without any log message. Any idea?

[Jordan Soo Yen Yih]

Hi Alvaro Cortes, did you try to use systemctl status clouseau.service to check the clouseau status? Btw, I ever experience that clouseau service slow to bootup due to server instance resources not enough, in this scenario, clouseau didn't throw any error message and when I checked on the CouchDB features list, there is no "search" feature show up. But after a time of period, It did appeared and was connected successful. I am not sure whether it is the same issue you have encountered, but you may try to use systemctl or check on CouchDB logs see any hints from there.