Don’t create the illusion of a job well done.
You've started maintaining a new project. You create a few tests, and they pass in your local dev environment. Then you deploy the code to production, only for the test to fail with a message "Couldn't find env var SOME_SERVICE_KEY".
Well… that env var exists in your local machine, and all other tests are passing. The pipeline should run the same way as you did locally. Why is the test you just wrote not working?
You ask a few senior devs, and they say in unison:
You have to rename the test file from test.spec.js to test.integration.js. That will fix the error.
You go back to the code, rename the file, commit, push to production, and everything works fine. Next time you write a test where you want to access some environment variable, you follow the guidelines and name the file *.integration.js so you never get this problem again.
However, the next dev may not know about this filename convention. The next dev may ask you the same thing you asked the senior devs. You decide to document this information in the developer's onboarding process:
Careful On Filenames
You have to rename test files as test.integration.js if you want to have access to test environment variables. Otherwise, the pipeline will fail, but the tests will work in your local machine.
Everybody is happy. Job well done! Tick.
Document the information so that the next developer knows how the system works the way it does. This way, they won't make the same mistake.
Imagine an alternative scenario.
You've started maintaining a new project. You create a few tests, and they pass in your local dev environment. Then you deploy the code to production, only for the test to fail with a message "Couldn't find env var SOME_SERVICE_KEY".
Well… that env var exists in your local machine, and all other tests are passing. The pipeline should run the same way as you did locally. Why is the test you just wrote not working?
You ask a few senior devs, one of them tell you this:
Let's pair. Here's how it works.
We use CircleCI.
When it downloads the project, CircleCI runs the file .circleci/config.yml.
One of the commands inside that CircleCI file is npm test.
npm test is an alias to npm test:spec and npm test:integration.
The command npm test:integration is an alias to . ./dev-environment.sh && mocha 'src/**/*.integration.js', which runs all tests with the integration suffix.
The ./dev-environment.sh file contains the value for all environment variables. One of those vars is "SOME_SERVICE_KEY".
In production, we use Elastic Beanstalk. The "SOME_SERVICE_KEY" var contains the value for production. In local development, "SOME_SERVICE_KEY" includes the value for a test key.
The reason why the error "Couldn't find env var" occurs in CircleCI but works in local development is that the test you created is reading "SOME_SERVICE_KEY", but the name of your file is *.spec.js, so it's running through npm run test:spec a command which doesn't inject . ./dev-environment.sh when it runs.
Why? We separate tests that rely on the environment, such as database credentials and third party API keys, from tests that don't. We call them Spec and Integration.
Why? We did this to prevent mixing deterministic tests with the ones that can have their result affected by environment-specific side-effects. The environment may be misconfigured, which doesn't mean the test is at fault, but rather the environment. We want to make sure most tests belong to a category where they are unlikely to fail due to the state of the environment. We want greater confidence that we've learned a major logical issue, not that we forgot to configure some software in our dev machine.
So you conclude from their explanation that you have to rename the file from test.spec.js to test.integration.js.
You go back to the code, rename the file, commit, push to production, and everything works fine. Next time you write a test where you want to access the environment variable, you name the file as *.integration.js, and you never get this problem ever again.
However, the next dev may not know about this filename convention. The next dev may ask you the same thing you asked the senior devs. You decide to document this information in the developer's onboarding process:
Testing convention
We separate tests that rely on the environment, such as database, env vars, and third party service calls, from tests that don't. We call them Spec (*.spec.js) and Integration (*.integration.js).
[…]
In the second example, they’re looking at the code and following it every step of the way together. As the more experienced dev walks through, they start to remember why the team wrote the code the way it is. It’s knowledge one wouldn’t acquire merely by trying to read the code themselves without assistance from someone who knows deeper.
Document the knowledge so that the next developer understands why the system works the way it does. This way, they won't make similar mistakes.
There's a long debate in academia between what constitutes information and what constitutes knowledge. The only thing they agree on is that information is a subset of knowledge. For this post, to give "information" is to tell the minimum amount of how it works (the how); to provide "knowledge" is to say the causes surrounding the effects and complement the information with the reasons why it works (the why).
Here are the tradeoffs for each scenario:
It's easier to pass information , either by text or words. However, it doesn't help the receiver solve other similar problems in the future without consulting the original source of information.
The developer has to ask the senior again for similar patterns in the code when there's an error. The developer has lost the ability to give feedback on the patterns and perhaps help the organization change them for the better. They're stuck on being aware of the minimum necessary to make it work instead of learning why it works that way.
It's harder to pass knowledge by text or words alone. It takes significant effort from the original source to pass knowledge than it is for information. That's why the senior dev proposed to sit together in pairing and show how and why it works, step by step.
Next time, the developer won't have to ask the senior for similar patterns in the code. For example, they may figure out by themselves that if they create a new UI test on the ui-tests folder, it will run on CircleCI automatically. They can learn it by themselves as they now have a broader understanding of how things work and why.
Like most programming, this is a short-term versus long-term tradeoff.
Although both result in the same outcome from an outsider perspective, they are far from being the same. While one may complain that the senior dev is spending too much time helping the new developer, another may complain that a handful of seniors are the bottleneck, so other developers need a broad understanding of the system. When you share knowledge, the Return On Investment is worthwhile.
Next time someone asks for your help, don't give them a mere piece of information. That's only going to turn you into the new Wikipedia among your peers. You've issued yourself a passport for the organization to promote you to your level of incompetence.
Next time you're facing a problem, don't be satisfied by a mere piece of information. You can only make an impact when you improve something while being aware of why people have done it the way it is, including the cause and effect surrounding it.
Encourage knowledge sharing instead of information sharing between all members of your teams. If you do, then you've got a high performing organization.
Share more knowledge, less information.
Now you know it.
Related Reading:
- guru99.com — Information vs Knowledge: Key Differences
- Braf, Ewa (2002), Knowledge or Information — What makes the difference? Centre/or studies on Humans, Technology and Organization
(CMTO). VITS Research Group, LinkOping University, Sweden - Dammann Olaf, MD, SM (2019). Data, Information, Evidence, and Knowledge: A Proposal for Health Informatics and Data Science. Online J Public Health Inform. 2018; 10(3): e224.
Thanks for reading. If you have some feedback, reach out to me on Twitter, Facebook or Github.
Top comments (0)