Packages, Lock files and Node Modules - A Deep Dive

Introduction

In software development, managing dependencies is a crucial aspect in building and maintaining projects. This article aims to provide a comprehensive understanding of package.json, package-lock.json, and node_modules, which are essential tools in ensuring the stability and reliability of a project. By the end of this article, you will have a clear understanding of how these tools work together, allowing you to avoid compatibility issues and reduce the risk of bugs.

Most importantly, you will never get the fear of "Why is my node_modules" is so huge?

So, let's get started, literally from scratch.

package.json

A package.json file can be created by running the command "npm init -y" in the terminal.

Let us create a new folder, and run the below command.

npm init -y

At this point of time, we will have a file with the name package.json

A typical package.json file contains

• name: the name of the project

• version: the version of the project

• description: a brief description of the project

• main: the main file of the project

• scripts: a list of scripts that can be run from the command line using npm

• dependencies: a list of packages that are required for the project to run

• devDependencies: a list of packages that are required for development, but not required for the production

Have you ever tried npm install on a package.json that has no dependencies?

If not, let's do it.

npm i

The work of the npm install is to install the dependencies that are listed in the package.json file. Since we do not have any, it does nothing. We do not get a folder with the name node_modules

Even though we do not have any dependencies in the package.json file, we will still get a package-lock.json but with the same information of package.json

Before we deep dive into the contents of package-lock.json let us break the misconception we have with node_modules - why it is so big, what are all those contents, where do they come from, etc.

Node Modules

In plain words, a module is a piece of reusable JavaScript code. It could be a .js file or a directory containing .js files. These modules make it easier for developers to create complex applications by breaking the code into smaller and more manageable components.

They come in three types

• Core Modules (http, os, fs, url, etc)

• Third-Party Modules (bootstrap, axios, styled-components etc)

• Local Modules

To understand how node_modules gets populated, let us take advantage of Third-Party modules and install them in different instalments.

Packages with No Dependency

We will be picking up two packages (Chalk, and JQuery) here which do not have any dependencies.

If you see the respective dependencies, you could see they have ZERO.

• Chalk - https://www.npmjs.com/package/chalk?activeTab=dependencies

• jquery - https://www.npmjs.com/package/jquery?activeTab=dependencies

Let's install these packages with the help of npm. Do remember our package.json is still the same with no dependencies

npm i chalk && npm i jquery

After this, you should see couple of things.

• chalk and jquery should appear in dependencies array

• Chalk and jquery node_modules

Now we can understand that node_modules is not just flooded with a bunch of packages, but it adds what we needed.

We will dive a bit further and understand what happens when a package has a dependency or a set of dependencies

For this case, let us take a famous package (axios) to understand.

npm i axios

If we check the dependency section of the axios, we could see axios has 3 dependencies

• follow-redirects

• proxy-form-env

• form-data

Out of these three, form-data has three more dependencies (asynckit, combined-stream, mime-types) further. And out of these three,

• combined-stream has a dependency - delayed-stream

• mime-types has a dependency - mime-db

Hence, our node_modules at this point in time would have installed a total of 9 packages and their respective folders inside the node_modules folder and it should look something like this.

This is it. Next time when we see a bunch of folders in node_modules, they are the packages that we need and their dependencies.

Since we have an idea of what a package.json file and node_modules folder does, let us move back to package-lock.json file.

package-lock.json

The package-lock.json is a file that is automatically generated by npm and it provides version information for all packages installed in a project, including all their dependencies.

This helps to ensure that a project works as expected even if a dependency has been updated since it was last installed.

The contents of the package-lock.json typically includes,

• name: the name of the project

• version: the version of the project

• lockfileVersion: the purpose of this file is to ensure that the dependencies and their sub-dependencies are installed consistently across different environments over time

• packages: lists the dependencies of a project and their specific versions. So when other developers install the project dependencies, they get the same versions.

• dependencies: a nested object that lists all the installed packages and their dependencies, along with their version numbers and other details such as integrity hashes, resolved URLs, etc.

"packages" in package-lock.json, is generated automatically which lists all the packages and their specific installed versions, including dependencies to ensure that the future installations of your project's dependencies are consistent, as it locks down the exact version of each dependency, rather than just specifying a version range as in the dependencies field.

Wrapping Up

In short, the integral parts of a Node.js project which helps to maintain the stablity of the projects are package.json, package.lock.json, and node_modules

• Do not forget to add node_modules in your .gitignore