DEV Community

loading...

Dependencies management in 2019: a review

elabftw profile image eLabFTW ・6 min read

dependencies

Hello folks,

The aim of this post is to make a (mostly) complete and accurate view of the different solutions available to you when you produce code and want to manage your dependencies like a boss, for different languages. We'll see how languages have different approaches to dependency management.

I'll be writing about these 9 languages:

  • PHP
  • Python
  • Rust
  • Go
  • Javascript
  • Elixir
  • Nim
  • C#
  • Ruby

Let's start with an easy one: PHP

Composer

GitHub logo composer / composer

Dependency Manager for PHP

Composer - Dependency Management for PHP

Composer helps you declare, manage, and install dependencies of PHP projects.

See https://getcomposer.org/ for more information and documentation.

Continuous Integration

Installation / Usage

Download and install Composer by following the official instructions.

For usage, see the documentation.

Packages

Find packages on Packagist.

Community

IRC channels are on irc.freenode.org: #composer for users and #composer-dev for development.

For support, Stack Overflow also offers a good collection of Composer related questions.

Please note that this project is released with a Contributor Code of Conduct By participating in this project and its community you agree to abide by those terms.

Requirements

PHP 5.3.2 or above (at least 5.3.4 recommended to avoid potential bugs)

Authors

See also the list of contributors who participated in this project.

Composer is the de facto standard for dependencies management in PHP. It's very hard today to get anything done in PHP without it. I recommend reading this article from Théo Fidry if you want to know more about the limitations of composer and the workarounds (like using PHARs).

PEAR

GitHub logo pear / pear-core

This is the definitive source of PEAR's core files.

PEAR - The PEAR Installer

https://travis-ci.org/pear/pear-core.svg?branch=stable

What is the PEAR Installer? What is PEAR?

PEAR is the PHP Extension and Application Repository, found at http://pear.php.net.

The PEAR Installer is this software, which contains executable files and PHP code that is used to download and install PEAR code from pear.php.net.

PEAR contains useful software libraries and applications such as MDB2 (database abstraction), HTML_QuickForm (HTML forms management) PhpDocumentor (auto-documentation generator), DB_DataObject (Data Access Abstraction), and many hundreds more Browse all available packages at http://pear.php.net, the list is constantly growing and updating to reflect improvements in the PHP language.

Warning

Do not run PEAR without installing it - if you downloaded this tarball manually, you MUST install it. Read the instructions in INSTALL prior to use.

Documentation

Documentation for PEAR can be found at http://pear.php.net/manual/. Installation documentation can be found in the INSTALL file included in this tarball.

Tests

Run the…

This is the old school player. It installs dependencies to the system. Only use it if your distribution is not shipping the extension in a package already. Note that you can use composer to install a PEAR package, see this part of phptherightway for more.

That's it for PHP. I'm pretty sure there are no other PHP dependencies management applications out there, but if I'm wrong please let me know in the comments section.

Second language: Python

Pip

GitHub logo pypa / pip

The Python package installer

pip - The Python Package Installer

https://readthedocs.org/projects/pip/badge/?version=latest

pip is the package installer for Python. You can use pip to install packages from the Python Package Index and other indexes.

Please take a look at our documentation for how to install and use pip:

We release updates regularly, with a new version every 3 months. Find more details in our documentation:

In 2020, we're working on improvements to the heart of pip. Please learn more and take our survey to help us do it right.

If you find bugs, need help, or want to talk to the developers, please use our mailing lists or chat rooms:

If you want to get involved head over to GitHub to get the source code, look at our development documentation and feel free to jump on the developer mailing lists and chat rooms:

pip is the goto package manager for python, it has its drawbacks but is a very popular tool to install Python dependencies. BTW, you should Stop using sudo pip install. Use pip with moderation (and the --user flag!).

Venv

The official module for managing your project's dependencies inside a virtual environment, thus not polluting the rest of the OS.

Virtualenv

GitHub logo pypa / virtualenv

Virtual Python Environment builder




This project started in 2011 and is very useful to isolate dependencies for a project. Part of this project is now venv in the standard python library. So you can think of virtualvenv as a more feature complete venv.

Pipenv

GitHub logo pypa / pipenv

Python Development Workflow for Humans.

Pipenv: Python Development Workflow for Humans

image image Azure Pipelines Build Status image


[ ~ Dependency Scanning by PyUp.io ~ ]

Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world Windows is a first-class citizen, in our world.

It automatically creates and manages a virtualenv for your projects, as well as adds/removes packages from your Pipfile as you install/uninstall packages. It also generates the ever-important Pipfile.lock, which is used to produce deterministic builds.

GIF demonstrating Pipenv's usage

The problems that Pipenv seeks to solve are multi-faceted:

  • You no longer need to use pip and virtualenv separately. They work together.
  • Managing a requirements.txt file can be problematic so Pipenv uses the upcoming Pipfile and Pipfile.lock instead which is superior for basic use cases.
  • Hashes are used everywhere, always. Security. Automatically expose security vulnerabilities.
  • Give you insight into your dependency graph (e.g. $ pipenv graph).

Pipenv started in early 2017 and was very quickly popular. It's my favorite, too. It works very similarly to composer with a lockfile and all the dependencies are installed in a special folder. To run the script in the environment, you need to prepend pipenv run. See my previous post about it. Note that the official Python doc is recommending pipenv for managing dependencies.

Something interesting to note, is that all of the above-mentioned projects are all hosted in the github.com/pypa namespace, the Python Package Authority, a group dedicated to maintaining the python packagers. I think it's nice to have this kind of "authority" for improving the ecosystem in Python.

The third language we'll talk about is: Rust

Cargo

Well, here it's a nobrainer, because (AFAIK) there is only one solution, and it's Cargo.

GitHub logo rust-lang / cargo

The Rust package manager

Cargo

Cargo downloads your Rust project’s dependencies and compiles your project.

Learn more at https://doc.rust-lang.org/cargo/

Code Status

Build Status

Code documentation: https://docs.rs/cargo/

Installing Cargo

Cargo is distributed by default with Rust, so if you've got rustc installed locally you probably also have cargo installed locally.

Compiling from Source

Cargo requires the following tools and packages to build:

  • git
  • curl (on Unix)
  • pkg-config (on Unix, used to figure out the libssl headers/libraries)
  • OpenSSL headers (only for Unix, this is the libssl-dev package on ubuntu)
  • cargo and rustc

First, you'll want to check out this repository

git clone https://github.com/rust-lang/cargo
cd cargo

With cargo already installed, you can simply run:

cargo build --release

Adding new subcommands to Cargo

Cargo is designed to be extensible with new subcommands without having to modify Cargo itself. See the Wiki page for more details and a list of known community-developed subcommands.

Releases

Cargo releases coincide with Rust releases High…

Cargo is great. Cargo is shit. YMMV. Honestly, it's pretty cool to have one official tool for dependencies management, at least it saves you the time you would have spent choosing one! It installs dependencies per-project but can also be used for system-wide packages. Interestingly, it uses the TOML syntax for the configuration file, whereas other projects chose YAML or JSON.

Let's move on to the fourth language: Go

go get is is the way to install a dependency in Go. But the issue is that it'll be installed system-wide (in $GOPATH). Say hello to conflicts and breaking changes because you can only have one version of a lib…

This led to other approaches like git submodules or project specific $GOPATH and a lot of developers being angry at Go.

However, with Go version 1.11 (released in August 2018) there is now something called modules. Check out the wiki page to know more about Go modules. I believe this will be the future of Go dependencies management, because fuck system-wide dependencies.

What do I see? Is that the fifth language? Yep: Javascript

Aaaaahhh, javascript and the dependencies, what a wonderful world! :)

node_modules

Npm

GitHub logo npm / cli

the package manager for JavaScript

npm(1) -- a JavaScript package manager

Build Status

SYNOPSIS

This is just enough info to get you up and running.

Much more info will be available via npm help once it's installed.

IMPORTANT

You need node v6 or higher to run this program.

To install an old and unsupported version of npm that works on node v5 and prior, clone the git repo and dig through the old tags and branches.

npm is configured to use npm, Inc.'s public registry at https://registry.npmjs.org by default. Use of the npm public registry is subject to terms of use available at https://www.npmjs.com/policies/terms.

You can configure npm to use any compatible registry you like, and even run your own registry. Check out the doc on registries.

Super Easy Install

npm is bundled with node.

Windows Computers

Get the MSI. npm is in it.

Apple Macintosh Computers

Get the pkg. npm is…

Everyone knows npm. You define your dependencies in a package.json and install them in the node_modules folder. Since version 5, there is a package-lock.json allowing you to have reproducible builds.

Yarn

GitHub logo yarnpkg / yarn

📦🐈 Fast, reliable, and secure dependency management.

Yarn

Fast, reliable, and secure dependency management

Circle Status Appveyor Status Azure Pipelines status Discord Chat Commitizen friendly


Fast: Yarn caches every package it has downloaded, so it never needs to download the same package again. It also does almost everything concurrently to maximize resource utilization. This means even faster installs.

Reliable: Using a detailed but concise lockfile format and a deterministic algorithm for install operations, Yarn is able to guarantee that any installation that works on one system will work exactly the same on another system.

Secure: Yarn uses checksums to verify the integrity of every installed package before its code is executed.

Features

  • Offline Mode. If you've installed a package before, then you can install it again without an internet connection.
  • Deterministic. The same dependencies will be installed in the same exact way on any machine, regardless of installation order.
  • Network Performance. Yarn efficiently queues requests and avoids request waterfalls in order to maximize network utilization.
  • Network Resilience. A single…

Yarn had reproducible builds well before npm. It also doesn't suffer from the sometimes poor decisions taken by npm's project leaders. It's fast, works well. It's my tool of choice.

Bower

Come on now! We said 2019 in the title! Forget about bower.

I will not talk about Parcel or Webpack as these are bundlers, not package managers.

Note that you can use either yarn or npm with the exact same package.json!

One thing to note about packages in javascript, is that it has become completely crazy, with micro libraries consisting of one line of code more or less, being used by hundred of thousands of other packages. For fun, go into a Javascript project and try this command:

ls node_modules | grep '^is*'

How fun is it to have is-obj, is-object, is-plain-obj, is-plain-object or things like isarray, but also is-arrayish… Do we really need a package is-windows when all it does is:

return process && (process.platform === 'win32' || /^(msys|cygwin)$/.test(process.env.OSTYPE)); ?

And good luck if you want to look at the code of all of these libraries for your project. That lead to the left-pad chaos.

So my advice here is to think hard before installing a javascript dependency, as it might come with hundred of these little library, and one day, one of them will become malicious (see malicious packages on npmjs.com).

You can use Depcheck to try and cleanup a bit your dependencies.

Let's talk about a more exciting language, like: Elixir

Elixir is a bit like Rust when it comes to dependency management: one official tool that gets the job done: Mix (or Hex for Erlang). Nothing more to say here, it works well enough and lets you focus on other things.

Okay so what do we have left? Nim?

Nim is a pretty new and cool language, and the name of the package manager is sooooo cute <3: Nimble. Which reminds me of this guy:

Nibbler

Nothing to see here, just use the one provided and get coding!

The next language is C#

Nuget

GitHub logo NuGet / NuGet.Client

Client Tools for NuGet - including Visual Studio extensions, command line tools, and msbuild support. (Open issues on https://github.com/nuget/home/issues)

NuGet logo


NuGet Client Tools

This repo contains the following clients:

Open Source Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Getting Started guide

For how to contribute to this repo follow the Contributing doc.

NuGet/Home repo

The NuGet/Home repo is the starting point for all things NuGet. It has the issue tracker and basic information about all things NuGet. Make sure to consult it before beginning your journey through NuGet code.

Feedback

File NuGet.Client bugs in the NuGet/Home.

License

Unless explicitly stated otherwise all files in this repository are licensed under the License in the root repository




This is the standard package manager that comes with Visual Studio. It exists since 2010 and is developed by the .NET foundation. I don't have much more to say about it because it's really windowsy and that's clearly not my platform of choice when it comes to anything vaguely related to computing.

Last but not least: Ruby

Rubygem

GitHub logo rubygems / rubygems

Library packaging and distribution for Ruby.

RubyGems Maintainability

RubyGems is a package management framework for Ruby.

A package (also known as a library) contains a set of functionality that can be invoked by a Ruby program, such as reading and parsing an XML file We call these packages "gems" and RubyGems is a tool to install, create, manage and load these packages in your Ruby environment.

RubyGems is also a client for RubyGems.org, a public repository of Gems that allows you to publish a Gem that can be shared and used by other developers. See our guide on publishing a Gem at guides.rubygems.org

Getting Started

Installing and managing a Gem is done through the gem command. To install a Gem such as Nokogiri which lets you read and parse XML in Ruby:

$ gem install nokogiri

RubyGems will download the Nokogiri Gem from RubyGems.org and install it into your Ruby environment.

Finally, inside your Ruby program…

This is the standard way to get dependencies for a ruby project. It can be seen as pip for ruby, because it installs packages system-wide. You can install them for your user with the --user-install flag (better: add gem: --user-install to ~/.gemrc to always install gems in your home).

So what about managing dependencies per-project you'll ask?

Bundler: a gem to bundle gems

GitHub logo rubygems / bundler

Manage your Ruby application's gem dependencies

Bundler is now maintained in the rubygems/rubygems repository.


Version     Inline docs Slack

Bundler: a gem to bundle gems

Bundler makes sure Ruby applications run the same code on every machine.

It does this by managing the gems that the application depends on. Given a list of gems, it can automatically download and install those gems, as well as any other gems needed by the gems that are listed. Before installing gems, it checks the versions of every gem to make sure that they are compatible, and can all be loaded at the same time. After the gems have been installed, Bundler can help you update some or all of them when new versions become available. Finally, it records the exact versions that have been installed, so that others can install the exact same gems.

Installation and usage

To install (or update to the latest version):

gem install bundler

To install a prerelease version (if…

You get a Gemfile and Gemfile.lock file, and you're sure the project will run everywhere with the same versions of gems.

But there is something interesting about the ruby ecosystem: it exists a lot more projects to switch between different ruby versions (chruby, rbenv, rvm, uru) than to install dependencies.

Conclusion

As you can see, some languages have it all figured out for you (rust, nim), while for others you need to make a choice (python, javascript), and for some you have system-wide and project-wide solutions (python, ruby). I don't quite understand why Go, a "recent" language didn't do a better job at dependency management. But this fits well with the view that "Go was created like we were still in the 70's".

Take home messages

  • Install your dependencies per project with a reproducible build (lock down versions of packages) to avoid surprises and to get something portable.

  • Try to avoid installing too many dependencies (javascript, I'm looking at you!).

  • Don't ship the dependencies with your project but ship the lockfile ;)

That's all folks, please leave a comment if you think I forgot something important or if I got something wrong ;)

Have fun coding!

Discussion (0)

pic
Editor guide