After a while – a new warning appeared:
Jenkins: your Jenkins data directory /var/lib/jenkins (aka JENKINS_HOME) is almost full
Checked disk usage with the
ncdu and found few jobs which used really lot of space:
root@jenkins-production:/data/jenkins# du -h --max-depth 1 jobs/ .. 25G jobs/APITests ...
And mainly space is used by a job’s logs:
root@jenkins-production:/data/jenkins# ls -lh /data/jenkins/jobs/APITests/jobs/Projectname/jobs/Stage/jobs/ProjectnameStageAPItests/builds/551/log -rw-r--r-- 1 root root 77M Mar 1 12:49 /data/jenkins/jobs/APITests/jobs/Projectname/jobs/Stage/jobs/ProjectnameStageAPItests/builds/551/log
There is a lot of old builds and each keeps its log.
The simplest way to solve this issue is to enable the Discard Old Builds option in a job’s config:
Apparently, it was added in Jenkins v2.х, as previously I remember had to manually describe a workspace cleanup in a Jenkinsfiles.
Check disk used now:
root@jenkins-production:/data/jenkins# du -h --max-depth 1 jobs/APITests/jobs/Projectname/ 23G jobs/APITests/jobs/Projectname/jobs 23G jobs/APITests/jobs/Projectname/
Save new settings, run a new build and check disk after it:
root@jenkins-production:/data/jenkins# du -h --max-depth 1 jobs/APITests/jobs/Projectname/ 7.0G jobs/APITests/jobs/Projectname/jobs 7.0G jobs/APITests/jobs/Projectname/
Also, there is the Discard Old Build plugin which allows to configure the cleanup process more accurately.
- 02/09/2017 Azure: подключение дополнительного диска к VM и миграция Jenkins (0)
- 02/08/2017 Jenkins: sandbox.RejectedAccessException: Scripts not permitted to use (0)
- 03/13/2019 Jenkins: import a job to another server (0)
- 03/19/2019 Jenkins: Scripted Pipeline – Production environment job confirmation step (0)