Down the Rabbit Hole Debugging Node.js Cipher Support

Ulisses Albuquerque — Tue, 04 Jun 2019 04:19:19 +0000

TL;DR: While most documentation on node.js and OpenSSL ciphers seem to indicate cryptographic algorithms are implemented in userland by OpenSSL, your Linux kernel version might impact the availability of some specific ciphers.

Recently while testing some code which leverages more recent cryptographic ciphers we discovered that node.js support for those is dependent on the node.js version, instead of completely relying on the underlying OpenSSL support.

With node.js 8.x this is what we get:

$ node -v
v8.16.0

$ node -e 'console.log(JSON.stringify(require("crypto").getCiphers()))'
["aes-128-cbc","aes-128-cbc-hmac-sha1","aes-128-cbc-hmac-sha256","aes-128-ccm",
"aes-128-cfb","aes-128-cfb1","aes-128-cfb8","aes-128-ctr","aes-128-ecb","aes-128-gcm",
"aes-128-ofb","aes-128-xts","aes-192-cbc","aes-192-ccm","aes-192-cfb","aes-192-cfb1",
"aes-192-cfb8","aes-192-ctr","aes-192-ecb","aes-192-gcm","aes-192-ofb","aes-256-cbc",
"aes-256-cbc-hmac-sha1","aes-256-cbc-hmac-sha256","aes-256-ccm","aes-256-cfb",
"aes-256-cfb1","aes-256-cfb8","aes-256-ctr","aes-256-ecb","aes-256-gcm","aes-256-ofb",
"aes-256-xts","aes128","aes192","aes256","bf","bf-cbc","bf-cfb","bf-ecb","bf-ofb",
"blowfish","camellia-128-cbc","camellia-128-cfb","camellia-128-cfb1",
"camellia-128-cfb8","camellia-128-ecb","camellia-128-ofb","camellia-192-cbc",
"camellia-192-cfb","camellia-192-cfb1","camellia-192-cfb8","camellia-192-ecb",
"camellia-192-ofb","camellia-256-cbc","camellia-256-cfb","camellia-256-cfb1",
"camellia-256-cfb8","camellia-256-ecb","camellia-256-ofb","camellia128","camellia192",
"camellia256","cast","cast-cbc","cast5-cbc","cast5-cfb","cast5-ecb","cast5-ofb","des",
"des-cbc","des-cfb","des-cfb1","des-cfb8","des-ecb","des-ede","des-ede-cbc","des-ede-cfb",
"des-ede-ofb","des-ede3","des-ede3-cbc","des-ede3-cfb","des-ede3-cfb1","des-ede3-cfb8",
"des-ede3-ofb","des-ofb","des3","desx","desx-cbc","id-aes128-CCM","id-aes128-GCM",
"id-aes128-wrap","id-aes192-CCM","id-aes192-GCM","id-aes192-wrap","id-aes256-CCM",
"id-aes256-GCM","id-aes256-wrap","id-smime-alg-CMS3DESwrap","idea","idea-cbc","idea-cfb",
"idea-ecb","idea-ofb","rc2","rc2-40-cbc","rc2-64-cbc","rc2-cbc","rc2-cfb","rc2-ecb",
"rc2-ofb","rc4","rc4-40","rc4-hmac-md5","seed","seed-cbc","seed-cfb","seed-ecb","seed-ofb"]

$ node -e 'console.log(require("crypto").getCiphers().length)'
119

However, when running the same code against node.js 10.x this is what we get:

$ node -v
v10.16.0

$ node -e 'console.log(JSON.stringify(require("crypto").getCiphers()))'
["aes-128-cbc","aes-128-cbc-hmac-sha1","aes-128-cbc-hmac-sha256","aes-128-ccm","aes-128-cfb",
"aes-128-cfb1","aes-128-cfb8","aes-128-ctr","aes-128-ecb","aes-128-gcm","aes-128-ocb",
"aes-128-ofb","aes-128-xts","aes-192-cbc","aes-192-ccm","aes-192-cfb","aes-192-cfb1",
"aes-192-cfb8","aes-192-ctr","aes-192-ecb","aes-192-gcm","aes-192-ocb","aes-192-ofb",
"aes-256-cbc","aes-256-cbc-hmac-sha1","aes-256-cbc-hmac-sha256","aes-256-ccm","aes-256-cfb",
"aes-256-cfb1","aes-256-cfb8","aes-256-ctr","aes-256-ecb","aes-256-gcm","aes-256-ocb",
"aes-256-ofb","aes-256-xts","aes128","aes128-wrap","aes192","aes192-wrap","aes256",
"aes256-wrap","aria-128-cbc","aria-128-ccm","aria-128-cfb","aria-128-cfb1","aria-128-cfb8",
"aria-128-ctr","aria-128-ecb","aria-128-gcm","aria-128-ofb","aria-192-cbc","aria-192-ccm",
"aria-192-cfb","aria-192-cfb1","aria-192-cfb8","aria-192-ctr","aria-192-ecb","aria-192-gcm",
"aria-192-ofb","aria-256-cbc","aria-256-ccm","aria-256-cfb","aria-256-cfb1","aria-256-cfb8",
"aria-256-ctr","aria-256-ecb","aria-256-gcm","aria-256-ofb","aria128","aria192","aria256",
"bf","bf-cbc","bf-cfb","bf-ecb","bf-ofb","blowfish","camellia-128-cbc","camellia-128-cfb",
"camellia-128-cfb1","camellia-128-cfb8","camellia-128-ctr","camellia-128-ecb",
"camellia-128-ofb","camellia-192-cbc","camellia-192-cfb","camellia-192-cfb1",
"camellia-192-cfb8","camellia-192-ctr","camellia-192-ecb","camellia-192-ofb",
"camellia-256-cbc","camellia-256-cfb","camellia-256-cfb1","camellia-256-cfb8",
"camellia-256-ctr","camellia-256-ecb","camellia-256-ofb","camellia128","camellia192",
"camellia256","cast","cast-cbc","cast5-cbc","cast5-cfb","cast5-ecb","cast5-ofb","chacha20",
"chacha20-poly1305","des","des-cbc","des-cfb","des-cfb1","des-cfb8","des-ecb","des-ede",
"des-ede-cbc","des-ede-cfb","des-ede-ecb","des-ede-ofb","des-ede3","des-ede3-cbc",
"des-ede3-cfb","des-ede3-cfb1","des-ede3-cfb8","des-ede3-ecb","des-ede3-ofb","des-ofb",
"des3","des3-wrap","desx","desx-cbc","id-aes128-CCM","id-aes128-GCM","id-aes128-wrap",
"id-aes128-wrap-pad","id-aes192-CCM","id-aes192-GCM","id-aes192-wrap","id-aes192-wrap-pad",
"id-aes256-CCM","id-aes256-GCM","id-aes256-wrap","id-aes256-wrap-pad",
"id-smime-alg-CMS3DESwrap","idea","idea-cbc","idea-cfb","idea-ecb","idea-ofb","rc2",
"rc2-128","rc2-40","rc2-40-cbc","rc2-64","rc2-64-cbc","rc2-cbc","rc2-cfb","rc2-ecb",
"rc2-ofb","rc4","rc4-40","rc4-hmac-md5","seed","seed-cbc","seed-cfb","seed-ecb","seed-ofb",
"sm4","sm4-cbc","sm4-cfb","sm4-ctr","sm4-ecb","sm4-ofb"]

$ node -e 'console.log(require("crypto").getCiphers().length)'
175

Because we were writing code in our local systems under node.js 10.x we were getting adequate coverage from our unit tests. However, once we started running the tests under our CI environment we got some errors. Turns out our CI environment does not have node.js 10.x available, only supporting node.js 8.x instead.

Leveraging nodenv we were able to run our code under node.js 8.x and identified the discrepancy shown above. We added some logic to our tests to skip the ones which touched node.js 10.x-specific ciphers. That made our tests pass in the CI environment, but the later Sonarqube quality gate which enforces test coverage now failed -- skipping non-available ciphers affected our coverage. Without a later version of node.js to use for testing in CI, we needed to change the way the tests were being run to ensure all code was being tested adequately.

Leveraging Docker

This is a somewhat common problem -- how to maintain test conditions as consistent as possible so you do not run into errors due to environmental differences. The solution is also pretty obvious -- we decided to use Docker images build on top of the official node base images. Our Dockerfile was quite simple:

ARG base_image
FROM ${base_image}

WORKDIR /opt/my-app-path
COPY . /opt/my-app-path
RUN npm install

CMD [ "npm", "test" ]

While there is definitely room for improvement (like using a non-root user, optimising for layer caching and more), it solves the key problem for us -- we can now build different versions of the image based on different versions of node.js by providing the base_image argument with all other libraries and binaries being the same across versions:

$ docker build \
  --build-arg base_image=node:8.16.0-stretch-slim \
  -t my-app:8.16.0-stretch-slim-latest

$ docker build \
  --build-arg base_image=node:10.16.0-stretch-slim \
  -t my-app:10.16.0-stretch-slim-latest

There were some additional hops to go through -- because the tests are now being executed inside a Docker container rather than directly in the build host, we need to mount an external path when running the tests and generate the results in a format our CI can parse.

$ docker run --rm \
  -v $(pwd)/test-output:/opt/my-app-path/test-output \
  my-app:8.16.0-stretch-slim-latest

We created a shell script which built test images for all the supported versions of node (8.x, 10.x and 12.x) and confirmed the correct ciphers were being skipped for version 8.x, but correctly used when running against 10.x and 12.x. We also stored test results in JSON files which included the version information alongside the test results, which could then be fed into plugins to our CI tool so we could get per-node-version test results. Everything looked good.

After committing the code, however, Sonarqube was still complaining about test coverage even on later versions of node.js. Clearly the test skip criteria was not behaving as expected in the CI environment -- something other than a node 10.x-specific cipher was not working as expected.

Digging Deeper

After adding some debugging code to the tests, including capturing the cipher list from both node.js and OpenSSL, we were able to pinpoint which algorithms were not available in the CI environment -- aes-128-cbc-hmac-sha256 which was being used with pbkdf2. Confusingly, though, when checking the cipher list for node.js inside the Docker image on our local systems, aes-128-cbc-hmac-sha256 was indeed included:

$ node -e 'console.log(JSON.stringify(require("crypto").getCiphers().filter(c => c.match(/aes-128-cbc/))))'
["aes-128-cbc","aes-128-cbc-hmac-sha1","aes-128-cbc-hmac-sha256"]

OpenSSL also indicated it was supported:

$ openssl list -cipher-algorithms | grep -i aes-128 
AES-128-CBC
AES-128-CBC-HMAC-SHA1
AES-128-CBC-HMAC-SHA256
AES-128-CFB
AES-128-CFB1
AES-128-CFB8
AES-128-CTR
AES-128-ECB
AES-128-OCB
AES-128-OFB
AES-128-XTS
aes128 => AES-128-CBC

Since Docker images are meant to abstract away environment issues, we were surprised to get distinct results when running the same commands in our CI environment -- aes-128-cbc-hmac-sha256 indeed was missing when running our tests on build agents.

When running containers, unless the user specifically exports host resources (like filesystem entries or ports) the only shared component between a Docker host and a container is the Linux kernel. That should not impact availability of ciphers, as OpenSSL implements all of its algorithms in userland code in the library... or does it?

That's when we came across the changelog for OpenSSL 1.1.0l, which includes the following tidbit:

  *) Added the AFALG engine. This is an async capable engine which is able to
     offload work to the Linux kernel. In this initial version it only supports
     AES128-CBC. The kernel must be version 4.1.0 or greater.
     [Catriona Lucey]

So, it turns out the Linux kernel version can indeed impact the availability of ciphers, or more specifically, of aes-128-cbc-hmac-sha256. That being said, the engine should be offered as an optimised implementation of the algorithm, not as the only one.

For now, we are continuing our investigation to determine whether this is expected behaviour for OpenSSL under Linux when using a pre-4.1.0 kernel.

DEV Community: Ulisses Albuquerque

Down the Rabbit Hole Debugging Node.js Cipher Support

Leveraging Docker

Digging Deeper