We’ve all heard about Gulp, Grunt, Webpack or whatever tool that's “cool” at the time you're reading this article. All these tools help with automating your project's build tasks in one way or another.
We all know that JavaScript is an interpreted language, so there is no actual code translation into bytecode done during this phase. However, it is also true that given JavaScript’s ecosystem, depending on the project, there can be quite a lot of tasks to perform prior, during, and after preparing our code for deployment. These tasks could include things like transpiling your TypeScript files into the actual JavasScript that will get executed, or your SCSS files into CSS, maybe even compile all your images into a sprite sheet for better web performance. And that is just to name a few, there are tons more.
With that being said, it is normal that many tools have been created during the past few years to help developers deal with the associated deployment pains. However, one particular tool that’s been there since the beginning has been vastly ignored. I’m referring to npm, and in this mastering npm tutorial, I’m going to show you a set of tips and tricks you can use to cover most of your orchestration needs with it and become an npm master. Please see this is a beginner's guide and if you are just getting started you are in the right place.
Let's start!
Tip No.1 - Initializing New Packages
To start you off with a simple one, you don’t need generators for your new projects. Yes, of course, if you’re planning on creating a new Express-based project, using Express’ own generator might be a good idea.
But if you’re starting something new, you might want to use npm’s own project generator. Yes, you read right, npm comes with a built-in generator, which helps you complete the basic information every npm project should have.
This is the command you should use:
your-folder> $ npm init
That’s it, simple, direct, to the point and you get the following as a response:
You basically get asked a set of questions to fill in your package.json file, which acts as a basic manifest for your project containing a basic description and some extra useful metadata about it e.g. the author’s name, what kind of license it’s being published under, and so on.
Note that some of the questions have an option between parenthesis. That means you can simply hit ENTER and that text will be entered by default, which comes extra handy when you’re not yet sure how to answer some of the questions.
Finally, at the end, you get a preview of how your new package.json file will look like. Then, hit ENTER again and it’ll be saved in your project’s folder.
Remember that anything and everything you’ve entered here can be changed in the future by editing the JSON file.
Tip No.2 - Freezing Dependency Versions
This is another simple, yet a forgotten feature. Did you know that if you install a new package without specifying it’s version, it’ll save it to your package.json file as version “^X.Y.Z”? Meaning, “at least, make sure you install version X.Y.Z of the package, but if there is a new one that matches X, then install that one” (so it could match 1.0.0 and 1.9.9 as well).
Maybe you don’t see a big issue with that, but in my years of working with Node.js, I can’t tell you how many times a poorly managed version change in one of my dependencies ended up preventing my code from working. It sounds crazy, but there is nothing out there that prevents OSS maintainers from making these types of mistakes. In fact, there was recently a problem with a hacker who gained access to the repository of a highly used package and after adding some malicious code, bumped the version up, so projects without version lock would install its new version without even asking for it.
I believe this default behavior from npm comes from an innate trust in OSS maintainers, but as we’ve come to learn in the past, this is not a very good practice. And you could be fooled into thinking this is easily fixed by removing the caret or any other symbol from the version number. That would tell the package manager to download exactly that version, and no other. In that case, the problem comes from your dependencies, if they’ve not done the same and version locked their own dependencies, then once you install them, you’ll run into the same problem.
So, how do we fix this problem? Enter npm shrinkwrap
$ npm shrinkwrap
The above command will rename your package.json file into npm-shrinkwrap.json, and inside the “dependencies” element, you’ll notice every single dependency the project and its dependencies have. And from now on, that file will be used whenever you run npm install. To give you an example, for a single dependency such as express, you end up with a 350+ lines JSON file. So yeah, there are a lot of dependencies to deal with.
You’ll notice that in your new JSON file, you’ll have a version that starts with “~”, which although not an exact match, will only allow updates on minor versions, so going back to the example, you could match 1.0.0 all the way to 1.0.X (with X being any valid number, of course). The point here is that minor version updates are only related to bug fixes that don’t break backward compatibility. With that being said, the version bump is still something that needs to be done manually, so if you want to be 100% sure, you should remove those “~” from versions everywhere.
Tip No.3 - Running Scripts
Let's now move on to more interesting orchestration opportunities, shall we? Running scripts is one of the key feats you can achieve with npm that will give you a chance to start worrying about Grunt, Gulp or any other alternative.
Thanks to this feature, you can even use npm as a language agnostic task runner! It’s as simple as doing this:
By adding the “scripts” element to your JSON file, you gain the ability to add extra functionality to npm. Out of the box, there are some commands that npm already expects such as start and test. So you can use those directly like this:
$ npm start
And the command you specify (node index.js in this example) will be executed. If on the other hand, you want to go ahead and add custom commands (which is the whole point of doing this), you can run them using npm run, like this:
$ npm run <command-name>
So, for example, on our screenshot, if we wanted to do the production preparations, we can do:
$ npm run prod-pred
And our prod-prep.sh script, is just a simple Bash script that could look like the following or as complex as you need it:
#!/bin/bash
echo "== Doing the production preps =="
echo "Running tests..."
mocha test
echo "Linting the code..."
jshint **.js
It’s a very simple mechanic, but powerful at the same time, without the need for an extra task runner. We can even use hooks to automate scripts depending on what we’re trying to do, so let’s look at that now.
You can also get the list of available commands by typing:
$ npm run
With no arguments, npm will return the full list of commands, grouped by “Lifecycle” related and those available through “npm run”, like this:
Lifecycle scripts included in test2-npm:
test
mocha test
start
node index.js
available via `npm run-script`:
prod-pred
sh prod-prep.sh
Tip No.4 - Running Scripts Before and After Other Scripts
Although running scripts is one of the most powerful features of npm, giving it the ability to come much more than simply a package manager. It goes one step further by providing you with hooks.
Long story short, every single command you can configure for your project will have a pre and post hook. Even those which you don’t have full control over, such as install, uninstall, publish or update. So you can do things like:
{
"name": "test",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"prepublish": "mocha test",
"build": "sh build-project.sh",
"postbuild": "rm -rf ./tmp"
},
"author": "",
"license": "ISC",
"dependencies": {
"mocha": "^5.2.0"
}
}
Notice how I’ve added a pre-publish hook, to make sure all tests pass before publishing the package (that single line will make sure you don’t forget about doing that ever again!). And I’ve also added a simple post-build hook, showing you how you can add hooks to your own commands as well, in this case, it will remove any temporary folder or files created by a generic build process.
The main takeaway here is that you can add hooks to existing and custom commands and the execution flow will look at their exit codes, meaning that if the tests fail for our first hook, the publish command will not be executed.
Tip No.5 - Automatically Increasing Package Versions
Another little-known feature of npm is that it allows you to automatically increase your package’s version number. The versioning system used by npm is semver, so you have three variants to run whenever you need to bump up your version number:
$ npm version patch
$ npm version minor
$ npm version major
To make sure we’re all on the same page: semver works by having 3 numbers for each version, distributed like this: major.minor.patch
And their meanings are:
Major: references the main package version number. Changes to this number imply that non-backward compatible changes have been made, either to a feature or to the entire thing. Either way, rest assured that if you were using it and jump to the next major version, unless you update your own code, something will break.
Minor: similar to the previous one, but a change in the minor version references a backward-compatible one. Meaning that most likely new features have been added to the package without breaking existing ones. If this number changes, it is recommended that you go through the documentation just to be sure.
Patch: as already discussed, changes to this number on the version reference small fixes, usually bug fixes.
Although it might seem like a small feature, you could pair it with scripts and hooks and come up with something like this:
{
"name": "test",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"publish:fix": "npm publish",
"publish:minor": "npm publish",
"publish:major": "npm publish",
"prepublish:fix": "npm version patch",
"prepublish:minor": "npm version minor",
"prepublish:major": "npm version major"
},
"author": "",
"license": "ISC",
"dependencies": {
"mocha": "^5.2.0"
}
}
And with that, you can simply choose what you’re publishing and forget about the version number. For example:
$ npm run publish:fix
Tip No.6 - Using npx
Sometimes your packages or others will add command line tools for you to use, I’ve been using the example of mocha, for instance, which adds the mocha command that needs to be executed in order to run and verify the tests.
Usually, these commands are either installed in the global cache or in the local node_modules folder, in which case they’re not readily accessible for you to use by simply typing their name.
Since version 5.2, npm added the npx command, which simplifies that task for you. Instead of having to know where the binary is located (if at all), it’ll look for it in your $PATH and then (if not found) in your local node_modules/.bin .
Not only that, but if it can’t find it anywhere, it’ll install the package for you and then run the binary, all in one step.
Using npx comes in handy when you need the power of a cli tool available from npm’s global registry, but you don’t want to have it installed on your system.
Tip No.7 - Adding npm Autocomplete to Your Bash
This one is strictly for *nix based systems or Bash for Windows 10, but I think it’s pretty useful to have under your tool belt.
One thing that is very useful when using Bash or any command line interface is the autocomplete feature they usually provide. The problem comes when you’re using custom command line tools, they don’t always provide auto-complete for their own arguments and (or) commands.
Luckily for us, npm allows you to add this feature to your bash, simply with a single line of code:
$ npm completion >> ~/.bashrc
Warning: Make sure you’re doing the double “>”, otherwise you’d be replacing your .bashrc file content, instead of adding the required code for autocomplete. After that, you can either open a new terminal or run:
$ source ~/.bashrc
That’ll reload your Bash configuration and the autocomplete with it. Once this is done, you can test it by typing npm ins and hitting TAB.
Note that the autocomplete feature also works for custom commands too, so you can also get that extra help if you’re heavy on the scripts!
Final tips
In this npm tutorial, I covered several ways to improve your "npm-fu", such as using scripts and hooks to run custom actions, automating your version handling and locking dependencies versions.
That being said, if you liked what you read, go check out npm’s official documentation, there are many things you can do with this tool that will probably help you get rid of some extra dependencies you might already have in your project.
Leave a comment if any of these tips where useful to you or send me a message on Hashnode if you have any questions about them!