Vue cli 2: Nightwatch + Browserstack
Recently, I had to configure the Nightwatch test we have on our Vue.js project to be able to run on Browserstack automated testing platform. This had to work for different environments. We have two main environments: test and prods.
The configuration we wanted to achieve was to be able to run the e2e test on:
- localhost using local selenium and Chromedriver.
- test/prod URLs using local selenium and Chromedriver
- test/prod URLs uring Browserstack
The final files are at the end of this article which you can paste from.
Initial setup
Our application is using Vue.js and was generated with vue cli 2 using the webpack template. We have a runner.js file, which is used to set up webpack, to be able to run the test on localhost. We will have to update this runner.js file and the nightwatch config.
Configuring Browserstack to only run on test/prod environments
Browserstack has a nice document on how to run Nightwatch test suite with them. The only problem is they assume you want to run on Browserstack all the time and do not show examples to have it configurable. What if you want to use a local version of Selenium and Chrome? It also gets more complex when you have to use Browserstack local to be able to access internal URLs, which was our case.
In order to run Browserstack local, you need to use a specific setup, which is set in a different runner file in their documentation.
1. Runner.js updates
In the runner, to use Browserstack local, we made it configurable using the command line node environment variable. We are expecting a variable we called RUNNER to be present, to specify which runner to use: local or Browserstack.
If the test needs to be run on Browserstack, we set up the runner to use Browserstack local. If not, we use the default configuration to set up the server, which is the same code provided by the Browserstack documentation.

2. Nightwatch config updates
Our config by default is using the local selenium. If the node environment variable RUNNERis provided to execute test on browserstack, we are overriding those settings to use Browserstack selenium (similar to the Browserstack documentation).

3. Update test status in Browserstack through API
By default, when executing the test, Browserstack won’t be able to show if a test failed. The only way to update this is by making a request to Browserstack API.
The request needs the session ID, which is available in the test or in the global afterEach executed at the end of each test suite. In the nightwatch config file, you can set this in globals property. If the test failed, we make a request to set them to fail. We also update the name to show of the session to display which environment the test was executed on: test or prod.

Final files
require('babel-register'); | |
const selenium = require('selenium-server'); | |
const chromedriver = require('chromedriver'); | |
const request = require('request'); | |
const config = require('../../config'); | |
const testConfig = require('../../config/test.env'); | |
const prodConfig = require('../../config/prod.env'); | |
const browserstackConfig = require('./browserstack.config'); | |
// http://nightwatchjs.org/gettingstarted#settings-file | |
const nightwatchConfig = { | |
src_folders: ['test/e2e/specs'], | |
output_folder: 'test/e2e/reports', | |
custom_assertions_path: ['test/e2e/custom-assertions'], | |
page_objects_path: 'test/e2e/pages', | |
selenium: { | |
start_process: true, | |
server_path: selenium.path, | |
host: '127.0.0.1', | |
port: 4445, | |
cli_args: { | |
'webdriver.chrome.driver': chromedriver.path, | |
}, | |
}, | |
test_settings: { | |
default: { | |
end_session_on_fail: false, // /!\ Important to get the session id back and update browserstack status on fail | |
selenium_host: '127.0.0.1', | |
selenium_port: 4445, | |
launch_url: `http://localhost:${process.env.PORT || config.dev.port}`, | |
silent: true, | |
desiredCapabilities: { | |
browserName: 'chrome', | |
javascriptEnabled: true, | |
acceptSslCerts: true, | |
}, | |
globals: { | |
...testConfig, | |
waitForConditionTimeout: 15000, | |
afterEach(client, done) { | |
if (process.env.RUNNER === browserstackConfig.RUNNER) { | |
if (client.currentTest.results.failed > 0) { | |
request({ | |
method: 'PUT', | |
uri: `https://api.browserstack.com/automate/sessions/${client.sessionId}.json`, | |
auth: { | |
user: browserstackConfig.USER, | |
pass: browserstackConfig.KEY, | |
}, | |
form: { | |
status: 'error', | |
reason: '', | |
}, | |
}); | |
} | |
const cliOptions = process.argv.slice(2); | |
const envIndex = cliOptions.indexOf('--env'); | |
const envName = cliOptions[envIndex + 1]; | |
request({ | |
method: 'PUT', | |
uri: `https://api.browserstack.com/automate/sessions/${client.sessionId}.json`, | |
auth: { | |
user: browserstackConfig.USER, | |
pass: browserstackConfig.KEY, | |
}, | |
form: { | |
name: `${envName} env: ${client.currentTest.module}`, | |
}, | |
}); | |
} | |
done(); | |
client.end(); | |
}, | |
}, | |
}, | |
headless: { | |
desiredCapabilities: { | |
browserName: 'chrome', | |
chromeOptions: { | |
args: [ | |
'--headless', '--no-sandbox', | |
], | |
}, | |
}, | |
}, | |
test: { | |
launch_url: 'https://test-env.url', | |
}, | |
prod: { | |
launch_url: 'https://prod-env.url', | |
filter: 'smoketests/*', | |
globals: { | |
...prodConfig, | |
}, | |
}, | |
}, | |
}; | |
if (process.env.RUNNER === browserstackConfig.RUNNER) { | |
nightwatchConfig.selenium = { | |
start_process: false, | |
host: 'hub-cloud.browserstack.com', | |
port: 80, | |
}; | |
const defaultTestSettings = nightwatchConfig.test_settings.default; | |
defaultTestSettings.selenium_host = nightwatchConfig.selenium.host; | |
defaultTestSettings.selenium_port = nightwatchConfig.selenium.port; | |
const browserStackDesiredCapabilities = { | |
'browserstack.user': browserstackConfig.USER, | |
'browserstack.key': browserstackConfig.KEY, | |
'browserstack.local': true, | |
browserName: process.env.BROWSER || 'chrome', | |
browser: process.env.BROWSER || 'Chrome', | |
chromeOptions: {}, | |
}; | |
nightwatchConfig.test_settings.test.desiredCapabilities = browserStackDesiredCapabilities; | |
nightwatchConfig.test_settings.prod.desiredCapabilities = browserStackDesiredCapabilities; | |
} | |
module.exports = nightwatchConfig; |
// 1. start the dev server using production config | |
process.env.NODE_ENV = 'test'; | |
const Nightwatch = require('nightwatch'); | |
const browserstack = require('browserstack-local'); | |
const webpack = require('webpack'); | |
const DevServer = require('webpack-dev-server'); | |
const spawn = require('cross-spawn'); | |
const browserstackConfig = require('./browserstack.config'); | |
const webpackConfig = require('../../build/webpack.prod.conf'); | |
const devConfigPromise = require('../../build/webpack.dev.conf'); | |
if (process.env.RUNNER === browserstackConfig.RUNNER) { | |
let bs_local; | |
try { | |
process.mainModule.filename = './node_modules/.bin/nightwatch'; | |
// Code to start browserstack local before start of test | |
console.log('Connecting local'); | |
Nightwatch.bs_local = bs_local = new browserstack.Local(); | |
bs_local.start({ key: browserstackConfig.KEY, forceLocal: 'true' }, (error) => { | |
if (error) throw error; | |
console.log('Connected. Now testing...'); | |
Nightwatch.cli((argv) => { | |
argv.config = 'test/e2e/nightwatch.conf.js'; | |
Nightwatch.CliRunner(argv) | |
.setup(null, () => { | |
// Code to stop browserstack local after end of parallel test | |
bs_local.stop(() => {}); | |
}) | |
.runTests(() => { | |
// Code to stop browserstack local after end of single test | |
bs_local.stop(() => {}); | |
}); | |
}); | |
}); | |
} catch (ex) { | |
console.log('There was an error while starting the test runner:\n\n'); | |
process.stderr.write(`${ex.stack}\n`); | |
process.exit(2); | |
} | |
} else { | |
let server; | |
devConfigPromise.then((devConfig) => { | |
const devServerOptions = devConfig.devServer; | |
const compiler = webpack(webpackConfig); | |
server = new DevServer(compiler, devServerOptions); | |
const { port, host } = devServerOptions; | |
return server.listen(port, host); | |
}) | |
.then(() => { | |
// 2. run the nightwatch test suite against it | |
// to run in additional browsers: | |
// 1. add an entry in test/e2e/nightwatch.conf.js under "test_settings" | |
// 2. add it to the --env flag below | |
// or override the environment flag, for example: `npm run e2e -- --env chrome,firefox` | |
// For more information on Nightwatch's config file, see | |
// http://nightwatchjs.org/guide#settings-file | |
let opts = process.argv.slice(2); | |
if (opts.indexOf('--config') === -1) { | |
opts = opts.concat(['--config', 'test/e2e/nightwatch.conf.js']); | |
} | |
const runner = spawn('./node_modules/.bin/nightwatch', opts, { stdio: 'inherit' }); | |
runner.on('exit', (code) => { | |
server.close(); | |
if (code !== 0) { | |
throw new Error('End to end tests failed...'); | |
} | |
}); | |
runner.on('error', (err) => { | |
console.log(`About to exit with code: ${err}`); | |
server.close(); | |
throw err; | |
}); | |
}); | |
} |
Thanks for reading, I hope this is helpful. I am planning on looking at how to do the same using Vue CLI 3 and write an article too. I will add a link here when it happens.
Update: here is the article with the Vue cli 3: https://medium.com/@digitaledawn/vue-cli-3-nightwatch-browserstack-76c61a0fbb79
Top comments (0)