“Please Be Cautious to Use Global Parameter” — A Warning for Developers
Developers sometimes use global parameters as a way to define and store a variable or constant value, which can then be used throughout the codebase. While this is a good way to make our codes efficient via saving hardware resources, developers should be aware of potential drawbacks to using global parameters and avoid using them in their projects as much as possible we can.
here are two cases partially caused by the inappropriate use of global parameters and appropriate solutions.
One scenario is that a global parameter in the file keeps the value temporarily to reduce Mongodb database connection checking in an AWS Lambda project. But when the AWS Lambda project shutdown and restart, it will copy the project in the previous container. So the global parameter won’t be changed once it’s given value.
One is that the developer is trying to define all the constants in the same file. The using function uses it with importing the unified constant defined file. However, ObjA is an object, when we pass the value of the ObjA property, the ObjA property in the cache will be changed. The next time, our function is running, the project is using the changed ObjA, not the original objA.
Case 1: a global parameter in the whole file of an AWS Lambda Project led to the wrong status of the MongoDB connection.
What happened and how did I analyze the process?
Our sms-blast function is reported by QA that didn’t work at all. By checking CloudWatch logs of the AWS Lambda, I find that there are so many connection timeout logs showing. So my first institution is the network between our AWS to MongoDB database isn’t working. And the unmatched VPC security policy with another health project also misled me that it’s the root cause. By restraining the project, it seems to be well
However, a week later, the QA relaunch the problem. I don’t know what happened, so I read the document about AWS Lambda and codes.
Our code is like this:
//index.js
const { connect } = require('./helpers/dbConnect');
const modelA = require('./helpers/modelA');
exports.handler = async (event, context) => {
context.callbackWaitsForEmptyEventLoop = false;
//executive connection
await connect();
for (const record of event.Records) {
const { body } = record;
const parsedBody = JSON.parse(body);
const message = JSON.parse(parsedBody.Message);
console.log(message);
if (!message) {
console.log('wrong message');
return;
}
const data = message.data;
//executive command:connection timeout error
modelA.update;
}
return {
statusCode: 200,
body: JSON.stringify('Message sent'),
};
};
//dbConnect.js
//the global parameter, error happened
let isConnected = 0;
module.exports = {
connect: async function () {
if (isConnected) {
console.log('already connectd…');
return;
}
var options = {
keepAlive: 1,
connectTimeoutMS: 30000,
useNewUrlParser: true,
readPreference: 'secondaryPreferred',
maxStalenessSeconds: 90,
poolSize: 100,
socketTimeoutMS: 2000000,
useUnifiedTopology: true,
};
mongoose.plugin(function (schema) {
schema.options.safe = {
j: 1, // the write must commit to the journal
wtimeout: 10000, // timeout after 10 seconds
};
});
mongoose.Promise = Promise;
try {
console.log('connecting to mongodb…');
const db = await mongoose.connect(process.env.MONGO_DB_URL, options);
isConnected = db.connections[0].readyState;
} catch (error) {
throw error;
}
},
};
So after observing the codes, the reference path is index -> connection->index-> execution. Does the error happen how to determine if it is connected? I find the previous developer used isConnected to store the value of the connection status.
But that ignores the connection of MongoDB has the expired time. The value in isConnected don’t have an expired time. it leads to the connection timeout when the command is executed.
Based on this, I’m not 100% confirming that’s the root cause. Because if every request is using the original codes, all things in theory should be successful until I got the rules of running AWS Lambda rules — t he Cold Start only happened in the first loading in a container. Only if the container is killed for no activities in a long time, the Cold Start will happen. Otherwise, the codes, especially the parameter are kept in the cache, also why it happens sometimes but not always.
Solutions:
Grounded on the analysis, we just need to have serval line codes changed to fix the problem below. Using the original property as the determination of the connection status.
//dbConnect.js
module.exports = {
connect: async function () {
//replace the global parameter with property
const isConnected = mongoose.connection && mongoose.connection.readyState === 1;
if (isConnected) {
console.log('already connectd…');
}
return;
var options = {
keepAlive: 1,
connectTimeoutMS: 30000,
useNewUrlParser: true,
readPreference: 'secondaryPreferred',
maxStalenessSeconds: 90,
poolSize: 100,
socketTimeoutMS: 2000000,
useUnifiedTopology: true,
};
mongoose.plugin(function (schema) {
schema.options.safe = {
j: 1, // the write must commit to the journal
wtimeout: 10000, // timeout after 10 seconds
};
});
mongoose.Promise = Promise;
console.log('connecting to mongodb…');
return mongoose.connect(process.env.MONGO_DB_URL, options);
},
};
After that, we observe the performance for 1 week, and all things are good.
The incorrect status and connection timeout errors resulted from the incorrect use of a global parameter to store the MongoDB connection status. The root cause was the MongoDB connection’s expiration time, which was not taken into account when using the global parameter. The solution was to change the parameter so that the original property determined the connection status.
Case 2: the shallow copy of the cached ObjA leads to the wrong data.
What happened and how did I analyze the process?
The data in my report feature is not accurate to the report by our QA. After the project runs locally we find the wrong code below.
//wrong code
…
const {optionalAdvancedFields} = require(`./${name}/constants`);
…
advancedFieldsMap[measure] = optionalAdvancedFields[measure];
//change advancedFieldsMap value
…
The root cause shows that the initial value importing from the external file was changed, but my design's purpose requires it won't be changed. Cuz the swallow copy easily happens in Object, so I think the key to this is the swallow copy during passing the value.
The deep thinking about why it happened, it’s similar to case 1. The requirement will load the cached file first and then load the file without the cache.
Solutions:
Based on the analysis, we have two ways to solve the problem.
- load the file every time, not the cache. But the rule of loading the stored first is to improve the efficiency of our running machine. so it will make our projects slower
//use delete require’s cache to forcefully load the constants from the original file, not recommended for low efficiency
...
const {optionalAdvancedFields} = require(`./${name}/constants`);
//fixed codes
delete require.cache[require.resolve(./${name}/constants)]
...
advancedFieldsMap[measure] = optionalAdvancedFields[measure];
//change advancedFieldsMap value
...
2. to avoid the swallow cache in concrete implementation. This way only affects the parameter without efficiency impact.
2.1 use JSON.parse() and JSON.stingy()
//1. use JSON.parse() and JSON.stingy()
...
const {optionalAdvancedFields} = require(`./${name}/constants`);
...
advancedFieldsMap[measure] =JSON.parse(JSON.stringy(optionalAdvancedFields[measure]));
...
2.2 use the function lodash npm package
import _ from 'lodash';
...
const {optionalAdvancedFields} = require(`./${name}/constants`);
...
advancedFieldsMap[measure] = _.cloneDeep(optionalAdvancedFields[measure]);
//change advancedFieldsMap value
...
2.3 create a factory function to resolve it
import _ from 'lodash';
...
const {optionalAdvancedFields} = require(`./${name}/constants`);
...
// factory.ts
export function getObjectTemplateData(param) {
return {...param}
}
import { getObjectTemplateData } from './factory.ts'
advancedFieldsMap= getObjectTemplateData(optionalAdvancedFields);
//change advancedFieldsMap value
A shallow copy of the cached object led to erroneous data in a report feature. The root cause was the requirement to load the cached file first, followed by loading the file without the cache. The solutions to this problem are to either load the file each time (which may impair efficiency) or to change the shallow copy to a deep copy using JSON.parse() and JSON.stringify() or the deep copy function in “lodash” npm package.
Conclusion：
In conclusion, the two cases presented here demonstrate the potential drawbacks of using global parameters in software development.
These cases highlight the importance of being aware of potential drawbacks when using global parameters and avoiding them in software development as much as possible. By doing so, developers can ensure the efficiency and accuracy of their code.
Reference:
https://aws.amazon.com/cn/blogs/compute/operating-lambda-performance-optimization-part-1/
Top comments (0)