Please, don't recommend pkg, or any bundling/packaging technique, as a security/privacy control.
If you need to protect sensitive/secret data (e.g. passwords, API tokens), you can use one of many symmetric (e.g. AES-256) or asymmetric (RSA) encryption algorithms.
Alternatively, there are developer tools that aim to solve this problem in a more developer friendly way than you having to manage public/private keys yourself. I personally like Hashicorp Vault.
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
You are right. Please let me point out that I was not recommending pkg as a security protocol but listing it as one of the reasons people ask me how to use it.
edit: additionally, is there any Windows equivalent of what you're doing in the example with strings?
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
Say Peter, how would you go about making your code unreadable then if this is not the way? Simply obfuscate it? That does not do a well enough job in my opinion.
If you're trying to make your code "unreadable", then obfuscation is what you're looking for. Keep in mind, obfuscation does not make your code "secure". There are such thing as deobfuscators.
If you want to "secure" your source code, well, there is little you can do in this area for the following reasons:
Dynamic languages are easily accessible/readable.
Compiled languages that compile to intermediate byte-code can be decompiled:
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
Hey Batman, are you on Windows or Linux? How are you building exactly?
From the docs: Just be sure to call pkg package.json or pkg . to make use of scripts and assets entries.
Also you might want to look at this Snapshot Filesystem part of the docs because maybe your assets are packaged correctly but you are not using the right path to access them.
If you want you can send me these files and I will have a look for you to see what is wrong. Skype jochem.stoel or Discord jochemstoel#7529
Thanks a lot. This was just what I needed. I need to distribute a cross platform utility including a small webserver, so node was an obvious choice. My only problem was that the users are mostly non-techs, so I didn't like the thought of them having to install node and all the dependencies. pkg works out of the box. I don't even need to create the package.json file and module exports and whatnot. I simply enter "pkg myutil.js" - done! A second after I have three executables, one for Linux, Windows and Mac.
Cross compile? I have a device my company still manufacturers and deploys world-wide, running Ubuntu 14.04.3 ... on an ARMv7 Processor. I have a node app I'm creating for the product family, and I'd like to run it on this device as well. Tried going the whole nvm route to install-and-run node directly on it, but gyphy fails to build some deps from the project locally on the device. I'd really much rather use pkg to build a binary to deploy to the device.
However, building the examples/express example from the pkg repo with pkg 4.4.9 like pkg . --targets node10.15.3-linux-armv7 --no-bytecode (on a linux box) and scp'ing the resulting binary over to the IOT device running the armv7 / Ubuntu 14 setup, I get the following error when trying to run the binary:
./express-example: relocation error: ./express-example: symbol
_ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE,
version GLIBCXX_3.4.21 not defined in file libstdc++.so.6 with link time reference
(Line wraps added to break long line)
Googling the error (specifically with regards to GLIBC and libstdc++.so.6) has gotten me nowhere. I can't figure out if the libstdc++ on the device is too old or too new. Tried updating libstdc++ but it said it was already at the latest version (for that OS.) I've got no clue where to go from here... Is there some way to compile the binary via pkg with different options, or statically link the libraries it needs instead of relying on system libraries?
Also, when I try to use a newer node version (like 10.21.0, etc) - it fails with an "unable to build" message. I know I can crosscompile regular C/C++ code on that linux box for ARM (we do that currently with Jenkins in the cloud on a linux box), so is there a way to get crosscompile working at buildtime?
Here's the error for building with 10.21:
[root@decidr express]# ./node_modules/.bin/pkg . --targets node10-linux-armv7 --no-bytecode
> pkg@4.4.9
> Fetching base Node.js binaries to PKG_CACHE_PATH
fetched-v10.21.0-linux-armv7 [ ] 0%
> Error! 404 Not Found
https://github.com/zeit/pkg-fetch/releases/download/v2.6/uploaded-v2.6-node-v10.21.0-linux-armv7
> Asset not found by direct link:
{"tag":"v2.6","name":"uploaded-v2.6-node-v10.21.0-linux-armv7"}
> Not found in GitHub releases:
{"tag":"v2.6","name":"uploaded-v2.6-node-v10.21.0-linux-armv7"}
> Building base binary from source:
built-v10.21.0-linux-armv7
> Error! Not able to build for 'armv7' here, only for 'x64'
I find myself rather stuck - can't run node directly on the device, and the device won't run the pkg-built binary, even though it builds ARMv7 code. No idea how to proceed forward - any assistance or ideas? :)
Re: libstdc++.so.6 and GLIBC
You can see what GLIBC versions are in libstdc++.so like this, with the right path to your libstdc++ file:
strings /usr/lib64/libstdc++* | grep GLIBC
The output on my system shows the highest version for C++ is 3.4.24
When I have had a problem before, it is usually one or two versions behind, with the compiler saying GLIBCXX_3.4.25 or .26 is needed.
GLIBCXX_3.4.22
GLIBCXX_3.4.23
GLIBCXX_3.4.24
GLIBC_2.2.5
GLIBC_2.3
Cofounded Host Collective (DiscountASP.net). Cofounded Player Axis (Social Gaming). Computer Scientist and Technology Evangelist with 20+ years of experience with JavaScript!
lol. I think it's a tongue in cheek jab at some of the comment discussions which have been... lengthy. Probably a discussion about my preferences to write function expressions instead of statements.
If you are curious, check it some of my articles. Most of them are pretty controversial. :)
I actually wouldn't find anything wrong with the code written here though.
And you wouldn't want to know how I would write it either. It'd probably involve pipes or compose or a new language spec I have been working on github.com/joelnet/MojiScript
But those things have their place. When the entire team understands FP. Or in your own personal projects etc. Always code to the team :D
I am trying to do this exact thing right now at work and just removed pkg because it doesn't support being behind a proxy - full stop, as far as I can tell. A work around mentioned on their github didn't work (just download the files to your cache, remove the failed file, retry), and I can't find any other solutions. A bummer because it seems like the big game in town.
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
The error message seems to be saying it can not execute/find powershell. Check the PKG docs for process.cwd() and how to deal with current working directory.
index.html not included in package error while running the express example
Could you help me on the example given at the pkg github page.
It keeps popping up this error when running the exercutable:
Error: File or directory '/**/express/views/index.html' was not included into executable at compilation stage. Please recompile adding it as asset or script.
at error_ENOENT (pkg/prelude/bootstrap.js:539:17)
at findNativeAddonForStat (pkg/prelude/bootstrap.js:1201:32)
at statFromSnapshot (pkg/prelude/bootstrap.js:1224:25)
at Object.stat (pkg/prelude/bootstrap.js:1250:5)
at SendStream.sendFile (/snapshot/express/node_modules/send/index.js:721:6)
at SendStream.pipe (/snapshot/express/node_modules/send/index.js:595:8)
at sendfile (/snapshot/express/node_modules/express/lib/response.js:1103:8)
at ServerResponse.sendFile (/snapshot/express/node_modules/express/lib/response.js:433:3)
at /snapshot/express/index.js:21:9
at Layer.handle as handle_request
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
Well, I can reach it, my "only" problem now is if I execute in Windows to create a executable windows file, it works. But if I create my exe in Linux, when I go to Windows Machine it doesn't work.
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
Does it throw an exception file not found when your run it? That might have something to do with the 'virtual' path your assets are stored. Those are not consistent on every platform.
Packaged files have /snapshot/ prefix in their paths (or C:\snapshot\ in Windows). If you used pkg /path/app.js command line, then __filename value will be likely /snapshot/path/app.js at run time. __dirname will be /snapshot/path as well.
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
Dependencies need to be in package.json
If you NPM install after you created your app, it will automatically add the dependency to package.json for you.
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
Lets say i have few executables app1.exe, app2.exe, app3.exe and i want to save up disk space.
So i would like pkg not to bundle node executable inside each binary, but dynamically link it. Is this possible ?
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
Well I'm still not sure entirely what you mean but you could create a single bundle of Node with a set of dependencies that you need, then let it execute process.argv[2] or start a REPL if none is provided.
What you basically have then is an executable that behaves just like Node.exe but with a few extra modules already included.
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
Yes, the bundled binary includes Node. It basically puts your scripts and the Node executable in one file.
As for loading Node dynamically, I think you are missing the point.
Pellentesque nec neque ex. Aliquam at quam vitae lacus convallis pulvinar. Mauris vitae ullamcorper lacus. Cras nisi dui, faucibus non dolor quis, volutpat euismod massa. Donec et pulvinar erat.
It's very easy to examine source code of bundled/packaged node.js applications.
Please, don't recommend
pkg, or any bundling/packaging technique, as a security/privacy control.If you need to protect sensitive/secret data (e.g. passwords, API tokens), you can use one of many symmetric (e.g. AES-256) or asymmetric (RSA) encryption algorithms.
Alternatively, there are developer tools that aim to solve this problem in a more developer friendly way than you having to manage public/private keys yourself. I personally like Hashicorp Vault.
You are right. Please let me point out that I was not recommending pkg as a security protocol but listing it as one of the reasons people ask me how to use it.
edit: additionally, is there any Windows equivalent of what you're doing in the example with strings?
The way you're presenting the topic implies that you're suggesting bundling/packaging applications for these use-cases.
superuser.com/questions/124081/is-...
Say Peter, how would you go about making your code unreadable then if this is not the way? Simply obfuscate it? That does not do a well enough job in my opinion.
It depends on what you're trying to accomplish.
If you're trying to make your code "unreadable", then obfuscation is what you're looking for. Keep in mind, obfuscation does not make your code "secure". There are such thing as deobfuscators.
If you want to "secure" your source code, well, there is little you can do in this area for the following reasons:
I don't know, never happened to me. I could have a look with you at your code if you want.
I don't know I'd have to see your code.
Hey Batman, are you on Windows or Linux? How are you building exactly?
From the docs: Just be sure to call pkg package.json or pkg . to make use of scripts and assets entries.
Also you might want to look at this Snapshot Filesystem part of the docs because maybe your assets are packaged correctly but you are not using the right path to access them.
If you want you can send me these files and I will have a look for you to see what is wrong. Skype jochem.stoel or Discord jochemstoel#7529
Thanks a lot. This was just what I needed. I need to distribute a cross platform utility including a small webserver, so node was an obvious choice. My only problem was that the users are mostly non-techs, so I didn't like the thought of them having to install node and all the dependencies. pkg works out of the box. I don't even need to create the package.json file and module exports and whatnot. I simply enter "pkg myutil.js" - done! A second after I have three executables, one for Linux, Windows and Mac.
Cross compile? I have a device my company still manufacturers and deploys world-wide, running Ubuntu 14.04.3 ... on an ARMv7 Processor. I have a node app I'm creating for the product family, and I'd like to run it on this device as well. Tried going the whole
nvmroute to install-and-run node directly on it, but gyphy fails to build some deps from the project locally on the device. I'd really much rather usepkgto build a binary to deploy to the device.However, building the
examples/expressexample from the pkg repo with pkg 4.4.9 likepkg . --targets node10.15.3-linux-armv7 --no-bytecode(on a linux box) and scp'ing the resulting binary over to the IOT device running thearmv7/ Ubuntu 14 setup, I get the following error when trying to run the binary:(Line wraps added to break long line)
Googling the error (specifically with regards to GLIBC and libstdc++.so.6) has gotten me nowhere. I can't figure out if the libstdc++ on the device is too old or too new. Tried updating libstdc++ but it said it was already at the latest version (for that OS.) I've got no clue where to go from here... Is there some way to compile the binary via pkg with different options, or statically link the libraries it needs instead of relying on system libraries?
Also, when I try to use a newer node version (like 10.21.0, etc) - it fails with an "unable to build" message. I know I can crosscompile regular C/C++ code on that linux box for ARM (we do that currently with Jenkins in the cloud on a linux box), so is there a way to get crosscompile working at buildtime?
Here's the error for building with 10.21:
I find myself rather stuck - can't run node directly on the device, and the device won't run the
pkg-built binary, even though it builds ARMv7 code. No idea how to proceed forward - any assistance or ideas? :)Re: libstdc++.so.6 and GLIBC
You can see what GLIBC versions are in libstdc++.so like this, with the right path to your libstdc++ file:
strings /usr/lib64/libstdc++* | grep GLIBC
The output on my system shows the highest version for C++ is 3.4.24
When I have had a problem before, it is usually one or two versions behind, with the compiler saying GLIBCXX_3.4.25 or .26 is needed.
GLIBCXX_3.4.22
GLIBCXX_3.4.23
GLIBCXX_3.4.24
GLIBC_2.2.5
GLIBC_2.3
see posts like stackoverflow.com/questions/447732... as sometimes the softlink without a version number points to an older file.
What are those "Yes, @joelnet " notes? Is there a reason for them?
lol. I think it's a tongue in cheek jab at some of the comment discussions which have been... lengthy. Probably a discussion about my preferences to write function expressions instead of statements.
If you are curious, check it some of my articles. Most of them are pretty controversial. :)
I actually wouldn't find anything wrong with the code written here though.
And you wouldn't want to know how I would write it either. It'd probably involve pipes or compose or a new language spec I have been working on github.com/joelnet/MojiScript
But those things have their place. When the entire team understands FP. Or in your own personal projects etc. Always code to the team :D
Yeah, from my perspective I guess it's some sort of inside joke between you both, but it's kind of mean if it isn't.
Also, we all know that joelnet would start a new line for each chained method :P
Nice article, for the rest, I didn't know about pkg :)
I am trying to do this exact thing right now at work and just removed pkg because it doesn't support being behind a proxy - full stop, as far as I can tell. A work around mentioned on their github didn't work (just download the files to your cache, remove the failed file, retry), and I can't find any other solutions. A bummer because it seems like the big game in town.
What exactly do you mean by not working behind a proxy?
I am facing the issue related to this. Updated the question on stackoverflow here :
stackoverflow.com/questions/546834...
Can You please help?
The error message seems to be saying it can not execute/find powershell. Check the PKG docs for process.cwd() and how to deal with current working directory.
index.html not included in package error while running the express example
Could you help me on the example given at the pkg github page.
It keeps popping up this error when running the exercutable:
Error: File or directory '/**/express/views/index.html' was not included into executable at compilation stage. Please recompile adding it as asset or script.
at error_ENOENT (pkg/prelude/bootstrap.js:539:17)
at findNativeAddonForStat (pkg/prelude/bootstrap.js:1201:32)
at statFromSnapshot (pkg/prelude/bootstrap.js:1224:25)
at Object.stat (pkg/prelude/bootstrap.js:1250:5)
at SendStream.sendFile (/snapshot/express/node_modules/send/index.js:721:6)
at SendStream.pipe (/snapshot/express/node_modules/send/index.js:595:8)
at sendfile (/snapshot/express/node_modules/express/lib/response.js:1103:8)
at ServerResponse.sendFile (/snapshot/express/node_modules/express/lib/response.js:433:3)
at /snapshot/express/index.js:21:9
at Layer.handle as handle_request
My issue is found here
What happens if your function have some environment variables (read from some file). How can you configure it in package.json?
You can include assets in your package too.
Well, I can reach it, my "only" problem now is if I execute in Windows to create a executable windows file, it works. But if I create my exe in Linux, when I go to Windows Machine it doesn't work.
Does it throw an exception file not found when your run it? That might have something to do with the 'virtual' path your assets are stored. Those are not consistent on every platform.
Possibly useful:
detecting assets
snapshot filesystem
Very interesting! How can i embed server dependencies like Express, Mongoose, etc?
I wrote this already.
I would like to brutally honestly point out that I have not really done much benchmarking to support my claim that there is no difference.
Can i use pkg not to include node in my binary but just link it from separate file ?
I don't understand.
Lets say i have few executables app1.exe, app2.exe, app3.exe and i want to save up disk space.
So i would like pkg not to bundle node executable inside each binary, but dynamically link it. Is this possible ?
Well I'm still not sure entirely what you mean but you could create a single bundle of Node with a set of dependencies that you need, then let it execute process.argv[2] or start a REPL if none is provided.
What you basically have then is an executable that behaves just like Node.exe but with a few extra modules already included.
Hope this helps you.
Bundled binary is large, it includes node itself ?
Is there a way to use pkg so it dynamically links to node instead ?
Yes, the bundled binary includes Node. It basically puts your scripts and the Node executable in one file.
As for loading Node dynamically, I think you are missing the point.
I would suggest using npx rather than installing it locally so you get the most up to date version.
What are you saying?
Hi, how can i run this exe everytime the user logs in ro windowz? It must run on every reboot.