Speed up Jenkins with npm build

Guixing Bai — Wed, 29 Nov 2017 11:24:37 +0000

Our team have many projects delivered by Jenkins. Every component need to be assembled by webpack which write in React.js or Vue.js.

Each build will spend 5 or 10 minutes to finish, and most time spend on npm install. But we all know that if package.json were not change, node_modules directory should not change as well.

So that we checksum package.json with MD5, and package whole node_modules directory as cache. If the hash string does not changed, which implied nothing changed, just unpack cached node_modules directory and then execute npm install, all things done.

Here is a simple example

PKG_SUM=$(md5sum package.json | cut -d\  -f 1)
NPM_TARBALL=node_modules-${PKG_SUM}.tgz
NPM_TARBALL_MD5SUM=${NPM_TARBALL}.md5sum
NPM_TARBALL_CACHE=${HOME}/.cache/npmtarball
S3_NPM_TARBALL=s3://deployment/npmtarball

[[ ! -e $NPM_TARBALL_CACHE ]] && mkdir -p $NPM_TARBALL_CACHE

function downloadNpmTarball(){
    pushd $NPM_TARBALL_CACHE
        if [ ! -f ${NPM_TARBALL} ];then
            s3cmd get ${S3_NPM_TARBALL}/${NPM_TARBALL} ${NPM_TARBALL}
            s3cmd get ${S3_NPM_TARBALL}/${NPM_TARBALL_MD5SUM} ${NPM_TARBALL_MD5SUM}
            md5sum -c ${NPM_TARBALL_MD5SUM} || rm -f ${NPM_TARBALL} ${NPM_TARBALL_MD5SUM}
        fi
    popd
}

function checkNpmMod() {
    downloadNpmTarball
    TARBALL=${NPM_TARBALL_CACHE}/${NPM_TARBALL}
    [[ -f $TARBALL ]] && tar xzf $TARBALL
}

function uploadNpmMod() {
    TARBALL=${NPM_TARBALL_CACHE}/${NPM_TARBALL}
    if [ ! -f  ${TARBALL} ];then
        if [ -d node_modules ];then
             tar zcf ${TARBALL} node_modules || return 1
        fi
        pushd $NPM_TARBALL_CACHE
             md5sum $NPM_TARBALL > ${NPM_TARBALL_MD5SUM}
             s3cmd put $NPM_TARBALL $S3_NPM_TARBALL/$NPM_TARBALL
             s3cmd put $NPM_TARBALL_MD5SUM $S3_NPM_TARBALL/$NPM_TARBALL_MD5SUM
        popd
    fi
}
checkNpmMod
npm install
uploadNpmMod

DEV Community: Guixing Bai

Speed up Jenkins with npm build