Building and testing Manticore Search

Contributing to a C++ project like Manticore Search can be a difficult task due to the language's low-level nature, which often necessitates specific versions of components and a thorough understanding of the system's internals. However, Manticore team has made this process significantly more accessible through the use of specialized Docker images. These images are carefully prepared to include all the necessary dependencies and configurations, making it possible for contributors to focus on their contributions without worrying about the underlying setup.

Contributing to Manticore Search can be streamlined through two powerful approaches:

• leveraging Docker for manual builds and tests
• and utilizing GitHub for automated processes.

This guide introduces both methods to suit contributors at different levels of the development cycle.

CI in your fork

An accessible way to begin contributing to Manticore Search is by forking the project on GitHub and enabling the Continuous Integration (CI) system configured within the project. This approach allows contributors to easily test their changes without the need to set up a local development environment. There's no need for specific CI runners; both the compilation of code for testing purposes and the building of packages for all supported operating systems are handled by GitHub's runners. Thus, you can easily replicate the entire CI process in your fork, with the sole exception of publishing packages to https://repo.manticoresearch.com/.

Getting Started with GitHub

1. Fork the Manticore Search GitHub repository to create a personal copy.
2. Clone your fork to your local machine using the command: git clone https://github.com/<your-username>/manticoresearch.
3. Enable Continuous Integration (CI) on your fork through the repository settings.
4. Push your changes to your GitHub repository. CI will automatically compile and test your code, providing feedback directly in your repository.
5. Create a pull request to the original (upstream) repository, requesting that your changes be merged.

This GitHub-centric approach simplifies the initial setup, allowing contributors to focus on development rather than environment configuration.

Building and testing manually with Docker

For those who prefer a hands-on approach or require more control over the build and test environment, such as remaking a specific test, there is an alternative solution. This involves manually using the same Docker images that the CI system employs to compile and test Manticore Search.

Compile Manticore Search with Docker

To compile Manticore Search, use the following Docker command in the root folder of your clone:

docker run -it --rm -e SYSROOT_URL=https://repo.manticoresearch.com/repository/sysroots \
-e arch=x86_64 \
-e DISTR=jammy \
-e boost=boost_nov22 \
-e sysroot=roots_nov22 \
-e CTEST_CONFIGURATION_TYPE=Debug \
-e NO_TESTS=1 \
-e CACHEB="../cache" \
-v $(pwd):/in/ \
manticoresearch/external_toolchain:clang16_cmake3263 bash

Note, you need to do it in the root folder of a clone from https://github.com/manticoresoftware/manticoresearch

After running this command, you'll be inside the container. Proceed with the following steps to compile the code:

1. Navigate to the mounted directory:

   cd /in/

1. Create a build directory and initiate the compile process:

   mkdir build
   ctest -VV -S misc/ctest/gltest.cmake --no-compress-output

The ctest -VV command triggers a verbose test mode, providing detailed output about the build process.

Testing Manticore Search

After compiling, you can test Manticore Search with another Docker command:

docker run --rm -it \
-e DIAGNOSTIC=1 \
-e NO_BUILD=1 \
-e WITH_COVERAGE=0 \
-e CACHEB="../cache" \
--name test \
-v $(pwd):/in/ \
manticoresearch/ubertests_ctest:3263_mar_2024 bash

Inside the container, run the following commands to start testing:

1. Navigate to the mounted directory:

   cd /in/

1. Remove the cmake cache left from the previous compilation:

   rm build/CMakeCache.txt

1. Execute the general test suite:

   ctest -V -S misc/ctest/gltest.cmake --no-compress-output

Optionally, you can run specific tests by including the CTEST_REGEX parameter:

   CTEST_REGEX=202 ctest -VV -S misc/ctest/gltest.cmake --no-compress-output

Running Ubertests

Another way of running a specific test is using the ubertest.php script directly (the above ctest command also executes it internally):

1. Navigate to the test directory:

   cd /in/test/

1. Run the ubertest script for specific tests:

   php ubertest.php -s /in/build/src/searchd -i /in/build/src/indexer t 202
   # Or for multiple tests
   php ubertest.php -s /in/build/src/searchd -i /in/build/src/indexer t 201 202

Conclusion

Both the GitHub and Docker ways offer distinct advantages for contributing to Manticore Search. The introduction of special Docker images at Manticore significantly reduces the complexity typically associated with contributing to C++ projects, enabling developers to focus more on their contributions and less on environment setup. Whether you prefer the simplicity and automation of GitHub or the detailed control with the Docker approach, these workflows facilitate efficient development and testing of your contributions to the project.