The problem
After playing around with using the jenkins:alpine base image and attempting to download a .war (60 MB) file using curl, I discovered that it was EXTREMELY slow. I am talking about the difference between it taking a minute and a half on my host machine and 15 mins during the docker image build.
FROM jenkins:alpine
ARG JENKINS_VERSION
ENV JENKINS_WAR="/usr/share/jenkins/jenkins.war"
USER root
RUN curl -o ${JENKINS_WAR} -L \
https://updates.jenkins-ci.org/download/war/${JENKINS_VERSION}/jenkins.war
RUN chown jenkins:jenkins ${JENKINS_WAR}
USER jenkins
Obviously, this is ridiculous and there had to be something wrong. After searching online, I found there was very little documentation on how to resolve this issue, so I started doing my own experimenting.
Overview
downloading .war of 60 MB took
- minute and a half on host machine
- 15 min during docker alpine image build
NOTICE
- made some updates were made on 12.4.2019
What did NOT work 👎
Adding docker build ... options
I was unable to get any increase from adding in options to docker build ... such as ...
-
--network host: performs the commands using the host network -
-m 1GB: allocates more memory to the image build
Fixing alpine DNS issues
I also read online about some DNS issues that alpine has and followed some of their instructions for fixing them. None of that helped increase download speeds.
Using the docker "ADD" command (12.4.2019)
Using ADD is easily the cleanest and most straightforward option. However, I found it was just as slow as using curl. See the secondary solution at the end of this article to see how I was able to still use the simplicity of ADD and while also getting an increase in speed.
FROM jenkins:alpine
...
ADD https://updates.jenkins-ci.org/download/war/${JENKINS_VERSION} ${JENKINS_WAR}
...
Changing image base (some success)
I noticed that when switched out from an alpine based image, I was able to significantly increase my download speeds. However, it still wasn't as fast as my host machine.
Notice the change in "TAG" in first line the FROM jenkins:TAG from "alpine" to "latest". The "latest" tag does not use alpine.
FROM jenkins:latest
ARG JENKINS_VERSION
ENV JENKINS_WAR="/usr/share/jenkins/jenkins.war"
USER root
RUN curl -o ${JENKINS_WAR} -L \
https://updates.jenkins-ci.org/download/war/${JENKINS_VERSION}/jenkins.war
RUN chown jenkins:jenkins ${JENKINS_WAR}
USER jenkins
Increasing curl limit rate (some success)
Using the alpine base image, I found that setting --limit-rate 1G as a curl ... option increased the download speed. I could have stopped at this point, but the download was still taking about 1-5 mins. It seemed rather unstable to me, so I kept experimenting.
FROM jenkins:alpine
ARG JENKINS_VERSION
ENV JENKINS_WAR="/usr/share/jenkins/jenkins.war"
USER root
RUN curl --limit-rate 1G -o ${JENKINS_WAR} -L \
https://updates.jenkins-ci.org/download/war/${JENKINS_VERSION}/jenkins.war
RUN chown jenkins:jenkins ${JENKINS_WAR}
USER jenkins
The solution 😄
I found that by combining switching the image base (from alpine to debian) and setting --limit-rate 1G curl option I was able to make the download happen in 2 seconds.
I still wanted to use alpine as my base image, so I changed the dockerfile to use multi-stage.
FROM debian:buster-slim as downloader
RUN apt update && apt install curl -y
USER root
ARG JENKINS_VERSION
WORKDIR /home/root
# running curl inside the debian image instead of alpine
RUN curl --limit-rate 1G -o jenkins.war -L \
https://updates.jenkins-ci.org/download/war/${JENKINS_VERSION}/jenkins.war
FROM jenkins:alpine
ENV JENKINS_WAR="/usr/share/jenkins/jenkins.war"
# getting the downloaded .war file from the debian image
COPY --from=downloader /home/root/jenkins.war ${JENKINS_WAR}
USER root
RUN chown jenkins:jenkins ${JENKINS_WAR}
USER jenkins
Another better solution 🚀 (12.4.2019)
It suddenly hit me that the ADD or COPY docker commands must have the capability to use URLs. Once I confirmed that ADD can do this, I felt so dumb. When I first tried using ADD to download the jenkins.war file it was still very slow 😢
BUT, then I thought, "what about switching the base image and then using ADD?". And what do you know, it was FAST 😄 It took about 3 seconds to download. This is more simple because the code is easier to follow and it removes the need to run apt update and apt install curl. This results in a smaller image overall.
FROM debian:buster-slim as downloader
ARG JENKINS_VERSION
WORKDIR /home/root
ADD https://updates.jenkins-ci.org/download/war/${JENKINS_VERSION}/jenkins.war \
jenkins.war
FROM jenkins:alpine
ENV JENKINS_WAR="/usr/share/jenkins/jenkins.war"
COPY --from=downloader /home/root/jenkins.war ${JENKINS_WAR}
USER root
RUN chown jenkins:jenkins ${JENKINS_WAR}
Top comments (1)
for me,
--cap-add=NET_ADMINresolved the issue