The Importance of Committing Lock Files in Dependency ManagementDependency Management Best Practices
In software development, dependency management is a critical aspect that directly impacts the stability and reproducibility of your projects. When working with package managers like npm, yarn, or others, one crucial decision developers must make is whether to commit the lock file to version control.
In this article, we’ll explore why it’s essential to commit lock files and how they help in ensuring deterministic, secure, and consistent builds.
A Bit of History
As you know,
npm was the default package manager for
Npm follows sem version system with wildcard support. That means if
foo is defined to use
^6.2.8, and you run
npm install, it will install the latest version of
Deterministic Builds and “It Works on My Machine” Conundrum
Imagine you are using
foo with version
^6.2.8 and you run
npm install. By default, npm installs the latest version,
6.2.9. Now, once this code is committed, other developers can check out the same codebase and proceed to execute
npm install. But this time it might install
^ always check for the latest non-major version.
In the ideal world,
6.2.10 should be compatible, but in the real world, that is not the case. These discrepancies can break your
build. Worse, it might work on your machine but break the production.
The Solution: Lock Files
Yarn first introduced the concept of lock files,
yarn.lock. It solves
the problem by locking the version of the package during initial installation. So,
^ will still install the non-major
version, but in the subsequent installations, it will use the same version as in the lock file.
How Lock Files Work?
npm install automatically generates the lock file.
Take a look at the below diagram.
Application A depends on libraries B (1.2.9) and C (^6.8.7). B depends on C (6.8.8), D (4.5.3), and E (7.2.9).
In the lock file, all the dependencies are flattened with their versions mentioned. So, the lock file will look like this B (1.2.9), C(6.8.7 and 6.8.8), D (4.5.3), and E (7.2.9), along with all the individual libraries dependencies mentioned and recursively so on.
Since C has been defined as
^, which means among 6.8.7 and 6.8.8, the 6.8.8 version will win. So, the next time you try to install dependencies, it won’t install 6.8.10 even if it has been released and meets the
The above description is a grossly simplified explanation of the process.
The Pitfalls of Using Exact Versions
Let’s eliminate wildcard characters (
*, etc.) and use the exact version. Wouldn’t that solve the problem?
Unfortunately, that’s not how it works. Even if you use the exact version for the library, the library itself might have dependencies with wildcard dependencies, and the application is still prone to breaking.
Lock files stores the integrity hash along with an exact dependency version, so it always checks whether the downloaded dependency is same as the one in the lock file.
Yarn introduced many features like caching, parallel downloads, etc. However, npm has caught up with most of these. And now, at the time of writing this, yarn and npm are almost the same.
Using a Lock File
Merely having a lock file isn’t sufficient. You need to make sure you execute the proper command(s).
npm install is intended to be used only in developer mode. If you want to install dependencies for production, you
need to use
List of commands with different modes for different tools
When Not to Commit Lock Files?
In cases where you’re building a library and publishing it, you don’t want to lock the dependencies because it might break the host application. Instead, let the host application handle locking to avoid potential conflicts. You can mark a range of versions your library supports.
Should You Change a Lock File Manually?
When facing issues caused by package updates, resist the temptation to manually adjust the lock file. It’s usually better to adapt your application to work with the latest version, or update your package.json to ensure correct lock file generation during the next installation.
Applicability Across Languages
In conclusion, committing lock files, along with executing the appropriate commands for dependency installation, is a best practice that ensures deterministic amd secure builds. By doing so, developers can mitigate issues related to version conflicts, enhance project security, and foster collaborative development in an environment of consistency and reliability.