SonarQube is a great tool for code quality and security checks. It is one of the tools we use to ensure quality of our products is measured and improved over time.
We also use GitLab – code management and CI/CD tool on some of our projects. SonarQube code analysis is integrated as a step in our GitLab CI pipelines.
For a long time I had a custom-built docker image for those purposes, automatically built and published to GitLab Container Registry.
Although quite automated, this custom-build docker image was adding to maintenance of the pipelines. I was happy to see that SonarSource has recently published official SonarScanner CLI docker image, and announced a support for GitLab integration in v8
SonarSource
/
sonar-scanner-cli-docker
Docker image for SonarScanner CLI
The docker image expects code being mounted on $PWD, which is set to /usr/src. This is different to where GitLab is mounting sources directory by default.
To make it all work the following environment variable needs to be set:
sonar:
image: sonarsource/sonar-scanner-cli
script:
- sonar-scanner -Dsonar.qualitygate.wait=true
allow_failure: true
variables:
SONAR_PROJECT_BASE_DIR: "${CI_PROJECT_DIR}"
Environment variable SONAR_PROJECT_BASE_DIR overrides default behavior. Predefined GitLab environment variable CI_PROJECT_DIR refers to the location where GitLab is mounting project source code.
Without this variable set the following misleading error will appear in GitLab log:
ERROR: Error during SonarScanner execution
java.lang.IllegalStateException: Unable to create user cache: /usr/src/.sonar/cache
at org.sonarsource.scanner.api.internal.cache.FileCache.createDir(FileCache.java:147)
at org.sonarsource.scanner.api.internal.cache.FileCache.<init>(FileCache.java:46)
at org.sonarsource.scanner.api.internal.cache.FileCache.create(FileCache.java:52)
at org.sonarsource.scanner.api.internal.cache.FileCacheBuilder.build(FileCacheBuilder.java:48)
at org.sonarsource.scanner.api.internal.JarDownloaderFactory.create(JarDownloaderFactory.java:42)
at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.createLauncher(IsolatedLauncherFactory.java:68)
at org.sonarsource.scanner.api.EmbeddedScanner.doStart(EmbeddedScanner.java:185)
at org.sonarsource.scanner.api.EmbeddedScanner.start(EmbeddedScanner.java:123)
at org.sonarsource.scanner.cli.Main.execute(Main.java:73)
at org.sonarsource.scanner.cli.Main.main(Main.java:61)
Caused by: java.nio.file.AccessDeniedException: /usr/src/.sonar
at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(Unknown Source)
at java.base/java.nio.file.Files.createDirectory(Unknown Source)
at java.base/java.nio.file.Files.createAndCheckIsDirectory(Unknown Source)
at java.base/java.nio.file.Files.createDirectories(Unknown Source)
at org.sonarsource.scanner.api.internal.cache.FileCache.createDir(FileCache.java:145)
... 9 more
ERROR:
ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.
Improvements
It might be a good idea to cache Sonar Scanner directories to be re-used in future pipeline runs:
sonar:
image: sonarsource/sonar-scanner-cli
script:
- sonar-scanner -Dsonar.qualitygate.wait=true
allow_failure: true
variables:
SONAR_PROJECT_BASE_DIR: "${CI_PROJECT_DIR}"
cache:
key: "sonar-${CI_PROJECT_ID}"
paths:
- ".scannerwork"
- ".sonar"
In my particular case, SonarQube analysis was not dependent on any other steps in the pipeline, so I've used needs and dependencies properties to make it run ahead of time and skip downloading artifacts of other steps.
sonar:
image: sonarsource/sonar-scanner-cli
needs: []
dependencies: []
script:
- sonar-scanner -Dsonar.qualitygate.wait=true
allow_failure: true
variables:
SONAR_PROJECT_BASE_DIR: "${CI_PROJECT_DIR}"
GIT_DEPTH: 0
cache:
key: "sonar-${CI_PROJECT_ID}"
paths:
- ".scannerwork"
- ".sonar"
Finally, I've used GIT_DEPTH environment variable to enforce Shallow Clone of our fairly large repository.
Latest comments (5)
I tried to use the above mentioned code and it worked perfectly fine. But after changing the CI_PROJECT_DIR when I am trying to scan my cpp files I can see INFO: CPD Executor Calculating CPD for 0 files. Can you please help on this.
I am using gitlab and created a property file
Hi @sachinkumar706
It is difficult to debug your setup with such minimal information on it, but doesn't look like GitLab is your issue at this stage.
I would highly recommend posting your question with property file and exact log details to community.sonarsource.com forums - there are a lot of good people to help there.
If I were to guess, I would suggest checking if you have
sonar.sourcesproperty set in yoursonar-project.propertiesfile. Make sure it points to your code directory (e.g../src).I am using Sonarqube first time and facing issue. Please help
sonar.projectKey=routing
sonar.projectName=cpp-lib-routing
sonar.projectVersion=1.0
sonar.sourceEncoding=UTF-8
sonar.scm.disabled=true
sonar.projectBaseDir=/builds/mappingfactory/cpp-lib-routing
sonar.sources=./src
sonar.scm.provider.disabled=true
sonar.scm.enabled=true
sonar.verbose=true
sonar.scm.provider=git
sonar.cfamily.build-wrapper-output=bw-outputs
sonar.reportPath=coverage.xml
18:00:41.198 INFO: Load plugins index (done) | time=54ms
18:00:41.247 INFO: Load/download plugins (done) | time=103ms
18:00:41.290 DEBUG: Plugins:
18:00:41.290 DEBUG: * Java Code Quality and Security 6.3.0.21585 (java)
18:00:41.290 DEBUG: * Code Smells 4.0.0 (smells)
18:00:41.290 DEBUG: * Git 1.11.1.2008 (scmgit)
18:00:41.291 DEBUG: * JaCoCo 1.1.0.898 (jacoco)
18:00:41.291 DEBUG: * SonarTS 2.1.0.4359 (typescript)
18:00:41.291 DEBUG: * Checkstyle 4.33 (checkstyle)
18:00:41.291 DEBUG: * SonarJS 6.2.1.12157 (javascript)
18:00:41.696 INFO: Process project properties
18:00:41.707 INFO: Process project properties (done) | time=10ms
18:00:41.711 INFO: Project key: routing
18:00:41.711 INFO: Base dir: /builds/mappingfactory/cpp-lib-routing
18:00:41.711 INFO: Working dir: /builds/mappingfactory/cpp-lib-routing/.scannerwork
18:00:41.711 DEBUG: Project global encoding: UTF-8, default locale: en_US
18:00:41.713 DEBUG: Creating module hierarchy
18:00:41.713 DEBUG: Init module 'cpp-lib-routing'
18:00:41.717 DEBUG: Base dir: /builds/mappingfactory/cpp-lib-routing
18:00:41.717 DEBUG: Working dir: /builds/mappingfactory/cpp-lib-routing/.scannerwork
18:00:41.718 DEBUG: Module global encoding: UTF-8, default locale: en_US
18:00:41.724 INFO: Load project settings for component key: 'routing'
18:00:41.747 DEBUG: GET 200 sonarqube.ci.mtp.cloud/api/setting... | time=23ms
18:00:41.750 INFO: Load project settings for component key: 'routing' (done) | time=26ms
18:00:41.838 DEBUG: Available languages:
18:00:41.838 DEBUG: * Java => "java"
18:00:41.838 DEBUG: * JavaScript => "js"
18:00:41.838 DEBUG: * TypeScript => "ts"
18:00:41.854 INFO: Load quality profiles
18:00:41.898 DEBUG: GET 200 sonarqube.ci.mtp.cloud/api/quality... | time=42ms
18:00:41.931 INFO: Load quality profiles (done) | time=77ms
18:00:41.941 INFO: Auto-configuring with CI 'Gitlab CI'
18:00:41.945 INFO: Load active rules
18:00:42.038 DEBUG: GET 200 sonarqube.ci.mtp.cloud/api/rules/s... | time=91ms
18:00:42.270 DEBUG: GET 200 sonarqube.ci.mtp.cloud/api/rules/s... | time=112ms
18:00:42.450 DEBUG: GET 200 sonarqube.ci.mtp.cloud/api/rules/s... | time=55ms
18:00:42.469 INFO: Load active rules (done) | time=524ms
18:00:42.475 DEBUG: SCM Step is disabled by configuration
18:00:42.503 DEBUG: Declared extensions of language Java were converted to sonar.lang.patterns.java : /*.java,/.jav
18:00:42.504 DEBUG: Declared extensions of language JavaScript were converted to sonar.lang.patterns.js : */.js,/.jsx,/.vue
18:00:42.504 DEBUG: Declared extensions of language TypeScript were converted to sonar.lang.patterns.ts : */*.ts,/*.tsx
18:00:42.506 INFO: Indexing files...
18:00:42.511 INFO: Project configuration:
18:00:42.527 DEBUG: 'src/contraction/routingkit_custom_contractor_hierarchy.cc' indexed with no language
18:00:42.529 DEBUG: 'src/contraction/routingkit_raw_mtp_graph_wrapper.cc' indexed with no language
18:00:42.531 DEBUG: 'src/CMakeLists.txt' indexed with no language
18:00:42.532 DEBUG: 'src/conversions.cc' indexed with no language
18:00:42.533 DEBUG: 'src/routing/cch_router_wrapper.cc' indexed with no language
18:00:42.534 INFO: 5 files indexed
18:00:42.535 INFO: ------------- Run sensors on module cpp-lib-routing
18:00:42.589 INFO: Load metrics repository
18:00:42.603 DEBUG: GET 200 sonarqube.ci.mtp.cloud/api/metrics... | time=14ms
18:00:42.628 INFO: Load metrics repository (done) | time=39ms
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by net.sf.cglib.core.ReflectUtils$1 (file:/builds/mappingfactory/cpp-lib-routing/.sonar/cache/52f5340dd05620cd162e2b9a45a57124/sonar-javascript-plugin.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of net.sf.cglib.core.ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
18:00:43.911 DEBUG: 'JavaSquidSensor' skipped because there is no related file in current project
18:00:43.912 DEBUG: 'Import external issues report' skipped because one of the required properties is missing
18:00:43.913 DEBUG: 'Import of Checkstyle issues' skipped because there is no related file in current project
18:00:43.913 DEBUG: 'Import of PMD issues' skipped because one of the required properties is missing
18:00:43.914 DEBUG: 'Import of SpotBugs issues' skipped because there is no related file in current project
18:00:43.914 DEBUG: 'SurefireSensor' skipped because there is no related file in current project
18:00:43.915 DEBUG: 'Removed properties sensor' skipped because there is no related file in current project
18:00:43.915 DEBUG: 'SmellMeasuresSensor' skipped because there is no related file in current project
18:00:43.915 DEBUG: 'SonarTS' skipped because there is no related file in current project
18:00:43.915 DEBUG: 'CheckstyleSensor' skipped because there is no related file in current project
18:00:43.916 DEBUG: 'JavaScript analysis' skipped because there is no related file in current project
18:00:43.916 DEBUG: 'SonarJS Coverage' skipped because there is no related file in current project
18:00:43.917 DEBUG: 'Import of ESLint issues' skipped because one of the required properties is missing
18:00:43.917 DEBUG: 'TypeScript analysis' skipped because there is no related file in current project
18:00:43.917 DEBUG: 'Import of TSLint issues' skipped because one of the required properties is missing
18:00:43.919 DEBUG: 'Generic Test Executions Report' skipped because one of the required properties is missing
18:00:43.920 DEBUG: Sensors : JavaXmlSensor -> JaCoCo XML Report Importer
18:00:43.920 INFO: Sensor JavaXmlSensor [java]
18:00:43.924 INFO: Sensor JavaXmlSensor java | time=4ms
18:00:43.925 INFO: Sensor JaCoCo XML Report Importer [jacoco]
18:00:43.927 INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
18:00:43.929 INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
18:00:43.929 INFO: Sensor JaCoCo XML Report Importer jacoco | time=4ms
18:00:43.933 INFO: ------------- Run sensors on project
18:00:43.937 DEBUG: 'Generic Coverage Report' skipped because one of the required properties is missing
18:00:43.938 DEBUG: 'Java CPD Block Indexer' skipped because there is no related file in current project
18:00:43.938 DEBUG: Sensors : Zero Coverage Sensor
18:00:43.938 INFO: Sensor Zero Coverage Sensor
18:00:43.940 INFO: Sensor Zero Coverage Sensor (done) | time=2ms
18:00:43.941 INFO: SCM Publisher is disabled
18:00:43.943 INFO: CPD Executor Calculating CPD for 0 files
18:00:43.946 INFO: CPD Executor CPD calculation finished (done) | time=0ms
18:00:43.957 DEBUG: SCM revision ID '78de2eb324e3c3546479b2c4afb34105e4242c9e'
18:00:44.029 INFO: Analysis report generated in 81ms, dir size=32 KB
18:00:44.043 INFO: Analysis report compressed in 14ms, zip size=4 KB
18:00:44.044 INFO: Analysis report generated in /builds/mappingfactory/cpp-lib-routing/.scannerwork/scanner-report
18:00:44.044 DEBUG: Upload report
18:00:44.069 DEBUG: POST 200 sonarqube.ci.mtp.cloud/api/ce/subm... | time=24ms
18:00:44.071 INFO: Analysis report uploaded in 27ms
18:00:44.073 DEBUG: Report metadata written to /builds/mappingfactory/cpp-lib-routing/.scannerwork/report-task.txt
18:00:44.073 INFO: ------------- Check Quality Gate status
18:00:44.073 INFO: Waiting for the analysis report to be processed (max 300s)
18:00:44.089 DEBUG: GET 200 sonarqube.ci.mtp.cloud/api/ce/task... | time=16ms
18:00:49.104 DEBUG: GET 200 sonarqube.ci.mtp.cloud/api/ce/task... | time=11ms
18:00:49.117 DEBUG: GET 200 sonarqube.ci.mtp.cloud/api/quality... | time=12ms
18:00:49.123 INFO: QUALITY GATE STATUS: PASSED - View details on sonarqube.ci.mtp.cloud/dashboard?i...
18:00:49.125 DEBUG: Post-jobs :
18:00:49.127 INFO: Analysis total time: 7.809 s
18:00:49.128 INFO: ------------------------------------------------------------------------
18:00:49.129 INFO: EXECUTION SUCCESS
18:00:49.129 INFO: ------------------------------------------------------------------------
18:00:49.129 INFO: Total time: 9.416s
18:00:49.161 INFO: Final Memory: 6M/34M
18:00:49.161 INFO: ------------------------------------------------------------------------
Running after_script
00:01
Saving cache
00:07
Creating cache sonar-6-2...
.scannerwork: found 9 matching files
.sonar: found 114 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Uploading artifacts for successful job
00:01
Job succeeded
18:00:43.943 INFO: CPD Executor Calculating CPD for 0 files
This is one error which I am getting
This is indeed setup-related SonarQube issue and not related to your GitLab setup.
Please refer to the SonarQube forums community.sonarsource.com and post your question there.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.