This feels like more of a build tool issue than a language issue.
I'll grant you: maven does a lot of stuff out of the box with very little configuration. The JS environment is definitely behind the curve here and a lot of that, in my opinion, is due to the vast array of options over the years (grunt, gulp, webpack, etc)
However, maven can also be a freaking nightmare once you have to deal with competing dependencies or if dependencies contain overlays.
None of this really contributes to the cognitive load of the language; rather, I think this speaks to the fact that the JS ecosystem hasn't yet reached the level of standardization that one experiences when dealing with Java.
I also think that the Java ecosystem has had the benefit of the attention of engineers with experience building large systems for longer and so have felt (and dealt with) the pain points that these entail.
I tend to agree. Although I have almost no Java experience, I can say that it's not all that hard to achieve the kind of all in one build process you're talking about in Node.js using NPM scripts and module loaders like webpack.
So - and I'm seriously interested here - how do you achieve:
Build all of the code
Run all of the unit tests
Run all of the Integration tests
Start up a temporary Postgres Database running the correct version
Set up the correct schema in the database
Start up the service being tested
Run all of the tests
Shut everything down
The Integration Tests part of that in particular I've always struggled with. The best I've been able to come up with is using Docker and Docker-Compose, but not only is that a bit flakey - getting the startup ordering right is non-trivial - but it's yet another tool for developers to need in order to run the full stack tests.
There are many ways to achieve this, and it does involve some initial work on the part of the developer, but once everything is setup, it is a thing of pure beauty. I'll give you an example of the build process for a small framework I'm working on, it might not touch on all the points you mention, but it should be enough to make you a (at least partial) believer. If you want to take a deeper look, check it out here.
So, with one command, git push here is what happens:
Runs all unit tests using jest
Transpiles TypeScript to JavaScript
Bundles up the code using rollup
Automatically generates API documentation using ts-doc
Pushes commits to the remote (github)
Starts up a build on Travis CI where it will:
Install all dependencies
Run all unit tests
Transpile TypeScript to JavaScript
Bundle up the code using rollup
Automatically generate API documentation using ts-doc
Send test coverage info to coveralls
Push API documentation to GitHub pages
Analyze the commits to see if there are changes that warrant a patch, minor or major version release.
If there is, modify package.json file to include new version number.
Push new release to NPM
Add release information to CHANGELOG.md
Commit the new package.json and CHANGELOG.md to GitHub remote.
Of course, your build process would not need many of these steps and would add some that aren't there. But the point is that, with one command, all these automated steps were triggered and you can make that happen too.
How? It all depends on the tools, libraries, services and environments that you use. But the basis for all this is NPM scripts, that you'll normally find in the package.json file. You might have to write a few custom Node scripts to do some fancier stuff but it's not all that complicated and is well worth it to automate the entire process. Hope this helps.
So, without wanting to sound flippant, what's the one command that someone who has only just checked out the repository can run to achieve all of that? And the one command that can be run before a commit - to make sure that I've not introduced any breaking changes in the commit I'm about to do?
It looks like it's npm run prepush, but it's not obvious to me.
And this is exactly my point. It's not obvious, because there is so much flexibility and so many different ways to achieve things.
In the Java/Maven world, I know that I can run mvn install on any project to run the complete build/test/etc lifecycle. This one command will:
Download all of the dependencies
Run all static checks
Perform any code generation steps needed
Perform any resource generation steps needed
Compile all of the code - main and test
Package everything up correctly
Run all of the tests - unit and integration
Additionally, if I ran mvn site then it will build the full set of documentation for the project. And if I instead ran mvn deploy then it will deploy the output to the configured destination - e.g. putting artifacts into Archiva, sending the actual application to Heroku, sending the built site to S3, etc.
And all of this works exactly the same on a single module or across a multiple-module build.
And the Integration tests can include steps to start and stop dependant services if needed - e.g. docker containers, the actual service under test, etc.
And packaging everything up can include into WAR files, executable JAR files, Docker containers, whatever is needed.
But the key thing is - I don't have to think about it. As someone new to the project, I already know how to work with it. As someone doing day-to-day work with it, it's easy.
(And, before you say, I know a bunch of that is because it's what I know, and that if I was more used to Node, or Python, or something like that then this would probably seem overly complex and that language would seem simpler.)
Super interesting. Like I said, I have next to no practical experience with Java, and the Java ecosystem, so it looks like I misunderstood exactly what you meant. What you are describing is very cool. I guess you summed it up nicely when you said:
there is so much flexibility and so many different ways to achieve things.
And that is a problem in the JavaScript universe, not only with build processes, but with pretty much every aspect of coding and development. It's the whole convention vs. configuration thing and in the JavaScript world, we are big time in the configuration side of things. There are so many ways to do everything. JavaScript fatigue is a real thing.
So you definitely have a point; you can't expect that running npm build on one JS project, will have the same impact as running npm build on another JS project. There is no convention when it comes to that.
I absolutely agree. However, for most people the build tool is their first interaction with a new project. And for the developers the build system is their day-to-day interaction. So a build system that makes life easy is hugely beneficial.
And then there's the fact that the build system is generally tied to the language. (Ignoring things like make...)
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
This feels like more of a build tool issue than a language issue.
I'll grant you: maven does a lot of stuff out of the box with very little configuration. The JS environment is definitely behind the curve here and a lot of that, in my opinion, is due to the vast array of options over the years (grunt, gulp, webpack, etc)
However, maven can also be a freaking nightmare once you have to deal with competing dependencies or if dependencies contain overlays.
None of this really contributes to the cognitive load of the language; rather, I think this speaks to the fact that the JS ecosystem hasn't yet reached the level of standardization that one experiences when dealing with Java.
I also think that the Java ecosystem has had the benefit of the attention of engineers with experience building large systems for longer and so have felt (and dealt with) the pain points that these entail.
I tend to agree. Although I have almost no Java experience, I can say that it's not all that hard to achieve the kind of all in one build process you're talking about in Node.js using NPM scripts and module loaders like webpack.
So - and I'm seriously interested here - how do you achieve:
The Integration Tests part of that in particular I've always struggled with. The best I've been able to come up with is using Docker and Docker-Compose, but not only is that a bit flakey - getting the startup ordering right is non-trivial - but it's yet another tool for developers to need in order to run the full stack tests.
There are many ways to achieve this, and it does involve some initial work on the part of the developer, but once everything is setup, it is a thing of pure beauty. I'll give you an example of the build process for a small framework I'm working on, it might not touch on all the points you mention, but it should be enough to make you a (at least partial) believer. If you want to take a deeper look, check it out here.
So, with one command,
git pushhere is what happens:Of course, your build process would not need many of these steps and would add some that aren't there. But the point is that, with one command, all these automated steps were triggered and you can make that happen too.
How? It all depends on the tools, libraries, services and environments that you use. But the basis for all this is NPM scripts, that you'll normally find in the package.json file. You might have to write a few custom Node scripts to do some fancier stuff but it's not all that complicated and is well worth it to automate the entire process. Hope this helps.
So, without wanting to sound flippant, what's the one command that someone who has only just checked out the repository can run to achieve all of that? And the one command that can be run before a commit - to make sure that I've not introduced any breaking changes in the commit I'm about to do?
It looks like it's
npm run prepush, but it's not obvious to me.And this is exactly my point. It's not obvious, because there is so much flexibility and so many different ways to achieve things.
In the Java/Maven world, I know that I can run
mvn installon any project to run the complete build/test/etc lifecycle. This one command will:Additionally, if I ran
mvn sitethen it will build the full set of documentation for the project. And if I instead ranmvn deploythen it will deploy the output to the configured destination - e.g. putting artifacts into Archiva, sending the actual application to Heroku, sending the built site to S3, etc.And all of this works exactly the same on a single module or across a multiple-module build.
And the Integration tests can include steps to start and stop dependant services if needed - e.g. docker containers, the actual service under test, etc.
And packaging everything up can include into WAR files, executable JAR files, Docker containers, whatever is needed.
But the key thing is - I don't have to think about it. As someone new to the project, I already know how to work with it. As someone doing day-to-day work with it, it's easy.
(And, before you say, I know a bunch of that is because it's what I know, and that if I was more used to Node, or Python, or something like that then this would probably seem overly complex and that language would seem simpler.)
Super interesting. Like I said, I have next to no practical experience with Java, and the Java ecosystem, so it looks like I misunderstood exactly what you meant. What you are describing is very cool. I guess you summed it up nicely when you said:
And that is a problem in the JavaScript universe, not only with build processes, but with pretty much every aspect of coding and development. It's the whole convention vs. configuration thing and in the JavaScript world, we are big time in the configuration side of things. There are so many ways to do everything. JavaScript fatigue is a real thing.
So you definitely have a point; you can't expect that running
npm buildon one JS project, will have the same impact as runningnpm buildon another JS project. There is no convention when it comes to that.I absolutely agree. However, for most people the build tool is their first interaction with a new project. And for the developers the build system is their day-to-day interaction. So a build system that makes life easy is hugely beneficial.
And then there's the fact that the build system is generally tied to the language. (Ignoring things like make...)