Which programming language has the best package manager?

Posted on Categories Continuous Updating, Package Manager

I have to work with a lot (9) of different package managers at my daily work at VersionEye. Part of our mission is it to make manual updating of dependencies extinct, because it’s a manual and time consuming task which nobody enjoys. That’s why we are building a notification system for open source software libraries to make Continuous Updating easy and fun. And since we support several programming languages – 8 at this point! – I get to write crawlers and parsers for all of them. To give you a better overview over the strengths and weaknesses of these package managers, I picked the most popular one for each language and will compare them. The contenders are:

  • RubyGems / Bundler (Ruby)
  • PIP / PyPI (Python)
  • Packagist / Composer (PHP)
  • NPM (Node.JS)
  • Bower (JS, CSS, HTML)
  • CocoaPods (Objective-C)
  • Maven (Java)
  • Lein (Clojure)

What are package managers?

Package managers are tools for software developers to help them easily share and consume software libraries and of course to manage dependencies! Specially transitive dependencies. If you are a software developer and you are still downloading software libraries with your browser and placing them into the right directory by hand, then you are doing something wrong! That is totally 1999!

Nowadays every software developer uses at least 1 package manager at work. That means you define the software libraries (dependencies) you want to use in your project in a project file. The package manager is then  responsible for

  • downloading the software libraries from a repository
  • placing the downloaded libraries into the right place and linking them correctly into the project
  • resolving transitive dependencies

A package manager always has 2 parts:

  • a Repository of binaries or source code
  • a client which communicates with the repository and performs some work on the client side. This is in most cases a command line tool

The advantages of such a system is pretty clear. It’s much easier to stay in control of dependencies since you just change a number in a text file and execute a command. The package manager is doing all the work for you. Another big advantage is that you don’t need to checkin the actual software libraries into your SCM. You only checkin the project file from your package manager.

RubyGems / Bundler (Ruby)

RubyGems is the package manager for Ruby. Libraries that are bundled as such are also called gems. A gem contains a directory with source code and a *.gemspec file. The *.gemspec file contains meta information to the gem, such as name, version, author, license and of course the dependencies of the gem. A gem can be build with the command line tool “gem” like this:

gem build rails.gemspec

And published to the central gem repository like this:

gem push rails-1.0.0.gem

In an non library (gem) project the dependencies are mostly managed by bundler, a very popular dependency manager for RubyGems. Bundler manages dependencies defined in a Gemfile, a simple text file. Here is an example:

source 'https://rubygems.org'

gem 'rails' , '3.2.16'
gem 'jquery-rails' , '2.2.1'

group :test do
  gem 'rspec' , '2.14.0'
end

The file starts with defining the repository. Which is rubygems.org, the central repository for all ruby gems. Each line in the file defines a gem, with name and version. Gems can be grouped together for different environments. E.g. gems in the test group are not required on production, but in test environments.

You don’t need to define an exact version number. You can also define a range of versions and take advantage of pre defined operators. Everything well documented on bundler.io.

Just execute the command in the directory where the Gemfile is placed and it will fetch all dependencies from the repository, resolves transitive dependencies and places them into the right directory.

bundle install

This command also creates or updates the file Gemfile.lock. This file contains the actually locked versions of the dependencies in the Gemfile and their transitive dependencies. The locked versions in the Gemfile.lock are specially important when you work with pessimistic version constraints. The Gemfile.lock is something you checkin to your SCM and never change by hand! More on that on bundler.io.

Pros

  • All gems are stored centralized on RubyGems.org. Really all ruby open source developers are using it and pushing their gems to it.
  • Learning curve is very low, very easy to understand how it works.
  • It’s very easy to publish new gems on rubygems.org. It takes less then a minute!
  • It has a very good REST JSON API.
  • Besides gems, a tag on a git repository can be defined as a dependency. That makes it very easy to fork & patch a software library.
  • Bundler supports semantic versioning and the pessimistic version constraints specifier enables to fetch the latest patches without breaking to support fetching always the newest patch/minor version of a gem.
  • There is a mirror at http://mirror1.prod.rhcloud.com/mirror/ruby/

Cons

  • Usually gems are not cryptographically signed! This can lead to security issues! It is possible to sign gems, but it’s not mandatory and most developers don’t sign their gems.
  • Defining a license for a gem is not mandatory. But most of the gems are using the MIT license.

You will not find much negative writing about it on the internet, besides the points which are listed here.

NPM (Node.JS)

NPM is the package manager for Node.JS. Libraries are stored as tgz files in the central Node.JS repository, which is npmjs.org. An NPM package is a zipped directory with source code and a package.json file, which contains all the meta information about the package, such as name, version, description, dependencies and so on. A package can be published like this:

npm publish

Here is an example for a package.json file from a well known Node.JS library. I modified the file a bit to get it shorter.

{
  "name": "request",
  "description": "Simplified HTTP request client.",
  "version": "2.31.1",
  "author": "Mikeal Rogers",
  "repository": {
    "type": "git",
    "url": "http://github.com/mikeal/request.git"
  },
  "bugs": {
    "url": "http://github.com/mikeal/request/issues"
  },
  "engines": [
    "node >= 0.8.0"
  ],
  "main": "index.js",
  "dependencies": {
    "qs": "~0.6.0",
    "json-stringify-safe": "~5.0.0",
    "forever-agent": "~0.5.0",
    "node-uuid": "~1.4.0",
    "mime": "~1.2.9"
  },
  "optionalDependencies": {
    "tough-cookie": "~0.10.0",
    "form-data": "~0.1.0",
    "tunnel-agent": "~0.3.0",
    "http-signature": "~0.10.0",
    "oauth-sign": "~0.3.0",
    "hawk": "~1.0.0",
    "aws-sign2": "~0.5.0"
  },
  "scripts": {
    "test": "node tests/run.js"
  }
}

By executing the following command in a directory with a package.json file, NPM will download all dependencies from the package.json file, resolve transitive dependencies and place them into the right place.

npm install

Pros

  • All packages are centralized at npmjs.org.
  • Learning curve is very low.
  • It’s very easy to publish new packages on npmjs.org.
  • It has a very good REST JSON API.
  • There is an NPM mirror in Europe.
  • Besides NPM packages, a tag on a git repository can be defined as a dependency. That makes it very easy to fork & patch a software library.
  • NPM supports semantic versioning and has an own operator which supports fetching always the newest patch/minor version of a package.

Cons

  • The NPM packages are not signed! That might lead to security issues!
  • Defining a license for a package is not mandatory.

NPM is a very young package manager. They learned from the failures of other package managers. It’s almost perfect!

Packagist / Composer (PHP)

Composer is the new package manager for PHP. It was written to replace PEAR. There are still less then 1.000 packages on Pear, but already more then 25K packages on packagist.org, the central repository for Composer packages.

Composer is similar to NPM. Dependencies are defined in a JSON file, called composer.json. Here is a very simple example:

{
  "name": "your-vendor-name/package-name",
  "version": "1.0.0",
  "require": {
    "php": ">=5.3.0",
    "another-vendor/package": "1.*"
  }
}

Executing this command in a directory with composer.json will install all dependencies into your project and generate a composer.lock file, same as in Ruby the Gemfile.lock.

php composer.phar install

One big difference to NPM is that packagist.org doesn’t host any files. It is completely based on Git/Svn/Hn tags. Submitting a package to packagist.org means submitting a link to a public Git, Subversion or Mercurial repository. packagist.org expects a composer.json file in the root directory of the master branch AND tags on the repository. The name of the tag is the version number displayed on packagist.org.

Pros

  • All packages are centralized at packagist.org.
  • Learning curve is very low.
  • It’s very easy to publish new packages.
  • It has a very good REST JSON API.
  • Licenses are mandatory.
  • It supports semantic versioning and has an own operator which supports fetching always the newest patch/minor version of a package.
  • Composer has a very unique and cool feature called “minimum stability”, which allows to specify the minimum stability for a requested dependency. More on that on the official documentation.

Cons

  • There are no mirrors. If packagist.org is down you can not fetch packages anymore.
  • The packages are not signed! It might lead to security issues!

The Composer / Packagist project is even younger then NPM. It is a big step forward for the whole PHP community. All big PHP frameworks moved already to Composer / Packagist.

PyPI (Python)

PyPI is the central repository for python packages. Dependencies are defined in a setup.py file, which is pretty much pure Python code. Here is an example:

 
from setuptools import setup
import os

setup(name='giki',
    version='0.1pre',
    description='a Git-based wiki',
    author='Adam Brenecki',
    author_email='adam@brenecki.id.au',
    url='',
    packages=['.'.join(i[0].split(os.sep))
        for i in os.walk('giki')
        if '__init__.py' in i[2]],
    install_requires=[
        'dulwich==0.8.5',
        'argparse', 'requests>=1.1.0,<1.3.0', 'colorama',
        'jinja2==2.6',
        'Werkzeug==0.8.3',
        'markdown2==2.0.1',
        'docutils==0.9.1',
        'textile==2.1.5',
    ],
    extras_require = {
        'test':  [
            'nose==1.1.2',
            'WebTest==1.4.0',
        ],
        'faster_markdown':  [
            'misaka==1.0.2',
        ],
    },
    entry_points = {
    'console_scripts':
        ['giki = giki.cli:main'],
    },
)

Installing the dependencies works like this:

python setup.py install

The packages at PyPI are hosted as “*.tar.gz” files. Each packages contains the source code and a setup.py file with meta informations, such as name, version and dependencies.

With a little bit preparation a package can be published to PyPI with this 2 commands:

 python setup.py register -r PyPI
 python setup.py sdist upload -r PyPI

Pros

  • All packages are centralized at PyPI.
  • Learning curve is very low.
  • It’s easy to publish new packages on PyPI.
  • There are multiple mirrors for fail-over scenarios.

Cons

  • The packages are not signed! That can lead to security issues.
  • Defining a license for a package is not mandatory.
  • No build-in support for semantic versioning.

PyPI is a robust package manager! There is room for improvements, that for sure. But the core commiters behind the project are aware of that and new features are in the pipeline.

CocoaPods (Objective-C)

CocoaPods.org is the central repository for CocoaPods, a package manager for Objective-C software libraries and mainly used by iOS developers. The CocoaPods command line tool is implemented in Ruby and hosted on RubyGems.org. It can be installed like this:

gem install cocoapods

Dependencies are defined in a simple text file called Podfile. Here a simple example:

platform :ios
pod 'JSONKit', '1.4'
pod 'Reachability', '3.0.0'

Executing this command in the root directory will install the dependencies:

pod install

Currently CocoaPods is completely relying on GitHub as backend. There is one CocoaPods/Specs repository with all Pod specifications available on CocoPods.org. Submitting a new pod package works via pull-request. Each pull-request gets reviewed and merged by a human after passing automated tests. This doesn’t scale infinitely but it scales for the current size and guarantees a high quality for the pod specs.

Pros

  • All packages are centralized at CocoaPods/Specs.
  • Learning curve is very low.
  • Very good documentation.
  • It’s easy to publish new packages, with a pull-request.
  • License information is mandatory! Pull-requests for Pods without license definition will not be merged.
  • It supports semantic versioning and has an own operator which supports fetching always the newest patch/minor version of a package.

Cons

  • The packages are not signed! That can lead to security issues.
  • No mirrors available.

CocoaPods is a young project. The quality of the Pod specs is very high, because of the human review. If they grow to the size of RubyGems this workflow will not scale anymore. But for right now it’s good enough.

Bower (Frontside JS & CSS)

Bower.io is a package manager for frontside JavaScript and CSS. Pretty much for everything what you are loading from the server to the browser. Packages like jQuery and Bootstrap for example.

The bower command line tool is available as package on NPM. It can be installed globally like this:

npm install -g bower

Installing a bower package works like this:

bower install <package>#<version>

It downloads the package and it’s dependencies into a directory called bower_components.

Every bower package is described in a bower.json file. Here’s an example:

{
  "name": "my-project",
  "version": "1.0.0",
  "main": "path/to/main.css",
  "ignore": [
    ".jshintrc",
    "**/*.txt"
  ],
  "dependencies": {
    "<name>": "<version>",
    "<name>": "<folder>",
    "<name>": "<package>"
  },
  "devDependencies": {
    "<test-framework-name>": "<version>"
  }
}

Bower is completely Git based and works without any user authentication. Everybody can register new packages like this:
bower register <my-package-name> <git-endpoint>
Bower expects the root of the git-endpoint to contain a bower.json file, describing the bower package. And it expects that there are some tags on the Git repository. The names of the tags are the version numbers.

To unregister a package you have to ask the maintainers in the longest GitHub issue in the history of mankind.

Pros

  • All packages are centralized at bower.io.
  • Learning curve is very low.
  • It’s easy to publish new packages. Works even without registration.

Cons

  • The packages are not signed! That can lead to security issues.
  • No mirrors available.
  • The process for unregistering a package is a No-Go.
  • License informations are not mandatory.
  • Many registered packages doesn’t provide a bower.json in their git repository. The package registration process doesn’t check if the package is valid, if it really contains a bower.json file or if there are some tags on the repository.

Bower is kind of cool, but the quality of the packages is many times bad. Unregistering a package is a pain. And a couple hundred registered packages even doesn’t provide a bower.json file in their repositories. User authentication and package validation would be necessary to improve this.

Maven (Java)

Maven is the package manager for Java. The central Maven repository is Search.maven.org.
A project and it’s dependencies are described in an pom.xml file. Here is an example.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

  <modelVersion>4.0.0</modelVersion>
  <groupId>org.ploin.pmf</groupId>
  <artifactId>ploinMailFactory</artifactId>
  <packaging>jar</packaging>
  <version>1.4.1</version>
  <name>ploinMailFactory</name>
  <url>http://www.ploinmailfactory.org</url>
  <description>Loadbalancing for Mail Sending</description>

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>${junitVersion}</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>commons-logging</groupId>
      <artifactId>commons-logging</artifactId>
      <version>1.1</version>
      <scope>compile</scope>
    </dependency>
  </dependencies>

</project>

Each dependency is identified by:

  • groupId
  • artifactId
  • version

Other properties like scope are optional. As you can see you need at least 5 lines to define 1 single dependency. Dependencies are checked, fetched, resolved and installed on each compile time.

mvn compile

The packages on the Maven Repository Server are pretty much maintained in a pre defined directory structure, based on groupId, artifactId and version of the artifact. The artifacts themselves are binaries, mostly *.jar files. You can see it on this mirror from ibiblio.

Publishing an artifact on the central maven repository takes only 1 week. No! I’m not kidding you! First of all you have to signup at JIRA and open a ticket. The whole process is described here. And be prepared to scroll 🙂

Pros

  • The artifacts are singed!
  • Licenses are mandatory for publishing!
  • There are many mirrors available all over the world.

Cons

  • Pushing artifacts to the maven central repository is not as easy as it should be. Because this process is a pain in the ass ALL big Java Frameworks and Companies are maintaining their own Maven Repository Servers. They push their artifacts first to their own MVN Server and then 1 or 2 weeks later the artifact appears on search.maven.org.
  • Not really centralized. Because of the previous point. There are at least 30 different public maven repository servers out there, who are not mirroring search.maven.org, but host other artifacts who are not available on search.maven.org. Java developers have to google for the right repository server with the desired artifacts.
  • Maven is a very complex tool! A pom.xml file can inherit from another pom.xml file. And a pom.xml file can include other pom.xml files. All that leads to high complexity and sometimes to resolution errors. Another side effect of this architecture is that resolving transitive dependencies this way is very slow, because different xml files have to be downloaded and parsed to build a complete model for 1 single artifact.
  • Maven violates the “Single Responsibility” pattern. Maven is not only doing dependency management! It is doing all kind of things which has nothing to do with dependency management. For example executing tests, generating reports, generating JavaDoc, doing deployments and many other things which are, in other languages, done by other tools.
  • Not sure why an artifact/package needs a “GroupId” AND an “ArtifactID” AND a version number. All other package managers are satisfied with a name and a version. KIS!
  • Having 5 lines of XML code for 1 single dependency is kind of overkill!

Maven is by far the most complex package manager I know. And know many of them!

Lein (Clojure)

Leiningen is the package manager for Clojure. It uses a maven repository as backend. Lein itself has the same scope as Maven. It’s not just doing dependency management! At the same time it’s a build tool. Dependencies are defined in a project.clj file. Here an example:

(defproject leiningen.org "1.0.0"
  :description "Generate static HTML for http://leiningen.org"
  :dependencies [[enlive "1.0.1"]
                 [cheshire "4.0.0"]
                 [org.markdownj/markdownj "0.3.0-1.0.2b4"]]
  :main leiningen.web)

Defining a dependency fits here into 1 line. And “GroupId” is not required for Clojure packages! Leiningen is using 2 sources for the dependency resolution.

The dependencies in a project.clj file can be explicitly fetched with this command:

lein deps

But they will be fetched anyway at compile time.

Publishing a package on clojars.org is pretty easy, after you signed up.

lein pom
scp pom.xml mylib.jar clojars@clojars.org:

Pros

  • The artifacts are signed!
  • It simplifies the dependency definition of maven.
  • It’s easy to publish new artifacts.
  • It’s easy to learn how it works.
  • It allows to reuse Java artifacts from other maven repositories.

Cons

  • It violates the “Single Responsibility” pattern. Same as Maven.
  • Licenses are not mandatory.
  • No Mirrors.

Leiningen is not perfect, but it is doing many things right.

Maven based package managers

There are many languages running on the JVM and most of them have their own package managers. But all of them are more or less based on Maven repositories. That means they all use a Maven Repository as backend and just have a better client (command line tool) and different conventions to deal with the backend.  These are the package managers from this family:

Ivy, SBT and Gradle support own repository types, beside Maven Repositories. But it kind of makes sense that they all access Maven Repositories, because they all want to reuse existing Maven artifacts.

Conclusion

All discussed package managers are open source and can be hosted internally as well. All of them are easy to get started with, except for Maven which is more complex and less fun. I’ve created a comparison matrix to highlight the differences with the most important attributes on the x axis. Unfortunately, there is no single package manager that offers every desired feature:

Comparison Package Managers

There is no clear winner and imho, there is no perfect package manager 😦 There are of course more package managers, such as CPAN (Perl) and Nuget (.NET), and we hope to be able to cover them in the future. You can find a complete list of package managers here. We are working continuously to add additional languages to our API and make VersionEye available for every software developer in the world!

What is your favorite package manager? And why? And which features are missing in current package managers? I’d love to hear your thoughts! Leave a comment or send me a tweet.

Follow the discussion on hacker news and Reddit.

if you read this far, you might want to monitor your packages on VersionEye 😉

106 thoughts on “Which programming language has the best package manager?”

  1. And then there would be perl and cpan which is mirrored, documented, tested, bugtracked, web-interfaced and commandlined, searchable and everything else…

    Oh, lots of modules in there, too. 😉

    Reply

  2. When you get around to also handling the grand-daddy of package managers (seriously, why did you leave out CPAN, one of the biggest and earliest ones?) please do add categories for testing and reporting of testing to a central infrastructure. Otherwise you’re missing out on some of the most crucial features of these installers.

    Reply

    1. Simply because of limited time and resources. We are currently only 3 devs at VersionEye. And we got more requests for Bower and CocoaPods. But CPAN is on our list as well. We will add it and it will be part of the next comparison.

      Reply

  5. With regard to your complaint about the dependency management tool (Maven/Leiningen) also being a build tool: be aware that the languages they are used for are compiled languages! When you use RubyGems to build and deploy a Gem, it does not need to download any of the dependencies because the language is not compiled. When building and deploying a JAR, however, it is necessary to download and use all of the dependencies in order to compile the code. Typically, a package manager is not needed in order to run an artifact that has been built. For example, once a WAR has been built, no more dependency resolution needs to be performed in order to deploy it to an app server. Zip assemblies are also used to produce artifacts that contain all the necessary dependencies in a ready-to-deploy-and-run package. If, however, that does not meet your needs there is still Ivy and Aether.

    I’m used to Maven, and having used Bundler there were a lot of frustrating things that I wished worked the same way as Maven. There is not a good equivalent to snapshot versions, for example. That leads you to needing to excessively increment version numbers manually or deleting gems from your gem server so the version can be re-pushed. Also, dealing with a project that has multiple gems worth of source code (eg. modularized rails apps) or multiple repositories can be annoying with Bundler.

    FYI: it appears Leiningen does support resolving artifacts from a mirror: https://github.com/technomancy/leiningen/issues/271

    Reply

    1. First to Leiningen. I didn’t say it can’t resolve artifacts from mirrors. Of course it can, because it’s Maven based. I’m just saying that there is currently on mirror for clojars.org. Please correct me if I’m wrong.

      I’m using Maven now since 8 years. And I still use it in active projects at VersionEye. For a compiled language it might make sense to combine dependency management and build tool. But that are not the only 2 jobs maven “tries” to do. Executing tests, generating reports and doing deployments is totally out of the scope. In other languages, not only dynamic ones, these kind of jobs are done by different tool. Specially in Ruby you have small tools which are doing 1 thing, but that they do right. And they can smoothly work together. But Maven tries to do everything. And I don’t think that that’s a good idea.

      Not everything that Maven does is bad. The signing and mirroring is a good feature. Also making the licenses mandatory is great. I would wish RubyGems would do it the same way. But the overall developer experience with bundler is still much better then with Maven. Even if it less secure.

      Reply

    2. Regarding SNAPSHOT versions:

      Bundler uses git semantics, as opposed to svn semantics, so the equivalent way to do a snapshot version is to have bundler track a snapshot branch. Of course, internally, git _always_ updates the version via the git commit hash, but I’d argue that that isn’t excessive.

      Firstly, Linux uses it – is your project seriously bigger than Linux?

      Secondly, if version semantics aren’t updated in any way or form when the code is updated, the only way to do a proper update is a complete re-sync of the upstream code. Maybe you’re used to that in the svn world, but that kind of roundtrip makes git people vomit.

      Reply

      1. I’m working on a project that is about an order of magnitude larger. It’s not exactly uncommon. It’s split up into “independent parts”, but the point still holds.

  6. Nuget and CPAN are very important. Sorry that they are not included this time. As soon Nuget and CPAN are integrated into VersionEye I will write a new comparison, where Nuget and CPAN are included as well. It’s just a madder of time and man power. We are not a big company and we didn’t had the resources to add them. But it’s coming!

    Reply

    1. Same story with bundler / RubyGems. The people don’t use it because it’s extra work and developers are lazy. It’s much more convenient not to do it. There is just one way out. The package manager has to enforce it. On maven all new submissions without signature getting declined. That’s the only reason why all Jar files on the official Maven repository are signed. But I didn’t saw one single self hosted maven repository there they sign their packages 🙂

      Reply

    1. I know that you can use composer with a self hosted version of packagist. But currently there is no official mirror of packagist.org. That means if packagist.org is down then you have no access to the 25K packages anymore. And there is no official mirror you could point your composer to in that case. Right? Or did I miss something?

      Reply

      1. You are right in there.

        Another case for satis is that it allows you to self-host, which is another point in your comparison.

      2. @Bram: The self-hosting column was very misleading. That did NOT mean that you can not host it by yourself. You can host ALL package managers here by yourself. It just means that the packages themselves are hosted by the package manager. That is not the case for packagist, because the downloads are mostly on GitHub. I talked with Nils Adermann about that point and I renamed the column to “Hosts downloads”. Hope that clears the confusion.

  11. To the point that Leiningen violates the “single responsibility principle”, because it handles the dependencies–this isn’t really the case; all the dependency handling is performed by the Aether library. Leiningen just hooks up a couple functions from that library to the config it reads from disk; there is no support in Leiningen itself for handling dependencies because the “single responsibility” it has is executing Clojure code, not managing packages.

    Reply

    1. Thanks for the hint. I didn’t know that. Is it possible to configure another library for the dependency management? I will take a closer look to the implementation.

      Reply

    1. Hi Allan. Good point. I took already a brief look to Dart and PUB. How popular is Dart nowadays in the Valley? You are the first one asking for Dart. It’s definitively on my radar, but further down the list. CPAN and Nuget are currently more up.

      Reply

  13. The process for deploying Maven artifacts to Maven Central is much simpler after the first time (when you have to register your groupId). I also find the groupId / artifactId / version scheme much better than the “flat” namespace that tools such as RubyGems offer much more scalable… no worries about your package name conflicting with some other package that’s out there. The XML syntax *COULD* use a bit of pruning, though.

    Reply

    1. You might be right with the deployment. But I think anyway it’s kind of over engineered. Every time somebody wants to push a new groupId, it gets reviewed by a human. Last time I did that, they asked me to change my groupId to the reverse domain of my server. And they checked if I’m really the owner of the server. That’s just to much control!
      That reminds my on the PEAR project. It worked the same way and the PHP community didn’t really adapt it. Than Packagist / Composer was coming around and it let everybody push what ever they want to push. 2 Years later there are 25K projects on Packagist and PEAR is almost dead.

      Having “long” or “flat” namespaces might be good or bad. But why not let decide the community? You can have long names on RubyGems and NPM as well. But obviously not many people desire that. First-come, first-served works pretty good. Even on GitHub with 6 million projects.
      What’s the point of forcing people to choose super long identifiers for little Jar files? First they force you to choose a groupId and then afterwards they start to change the namespaces because it doesn’t fit in their logical order. Best example is the spring framework. First the groupId was “spring”, then it was “springframework” and now it’s “org.springframework”. Maybe next year it’s “com.springframework” because it’s a profitable company. Who knows?

      XML was hip … 10 years ago. Maven is the only package manager which is still using XML as configuration. And because Maven is doing many things at the same time the pom.xml files are growing like crazy. I saw already pom.xml files with 2000 lines. And now tell me that it’s better then Ant!? 🙂

      Reply

    1. The column “Self hosted” in the infographic is misleading. I doesn’t mean that you can self host it or not. It means that the artifacts are hosted on the repository and are not just links to git tags. I just talked with Nils Adermann about that and he recommended to rename the column to “hosts downloads”.

      I know that you can mirror packagist. But is there currently an official mirror of packagist? If packagist.org goes down now, is there an alternative mirror with all meta informations?

      Reply

  16. I notice that you haven’t included Nuget, which is a pretty good package manager for C#. Is this List exclusively for open tools ? Even if it is Nuget being open source is a valid contender. just my 2 cents

    Reply

    1. Hi Vivek. I didn’t explicitly exclude Nuget! There are many other application level package managers like Nuget and CPAN. I just didn’t had time to check them out closely. But I will do it in the next months and in the next comparison Nuget will be included.

      Reply

      1. Thanks Robert, Looking forward to it! Now if you’ll excuse me I’ll have to the get my beer promised to me by Microsoft 😉

  17. If you intend to investigate more package managers, I’d like to raise your attention towards Quicklisp, which is the package manager for Common Lisp, and which has been extremely useful in helping both newcomers and old-timers find and install libraries. Before QL, this was an incredibly cumbersome process which has now been replaced by a single command. More information here: http://www.quicklisp.org/

    Reply

  18. Has anybody ever tried to solve this problem in a language independent fashion?

    To a very limited extent, RubyGems is used outside of Ruby. There are plenty of “Ruby” gems out there that are essentially just wrappers around a JavaScript or CSS library, allowing for that library to be included in Rails’ asset management.

    Reply

    1. Very good point. A language agnostic package manager would make my life much easier 🙂

      As far as I know there is currently no language agnostic package manager. I know that Sonatype tries to integrate Ruby into their Maven Repository System (Java). Another good candidate would be npmjs, because they store packages as tgz files, basically simple zip files. Currently the zip files contain JavaScript code and a json file with meta information. I would prefer NPM or RubyGems because they are both less complex then Maven.

      Reply

  19. Gradle and IVY don’t require a maven backend. The can consume it, but they don’t require it. There are seperate Gradle and IVY backends, which are similar to Maven backends, but not quite the same. IVY handles SNAPSHOTS much different then Maven for example. Gradle was originally based on Ivy, but has since moved on to it’s own slightly different backend.

    Reply

    1. I just added an additional sentence for that to the article. Sure Ivy and Gradle can handle different repo types. But in the official Ivy documentation, the quick starter they write: “Ivy uses the maven 2 repository to resolve the dependencies you declare in an Ivy file”. And somehow that makes sense because you want to take advantage of existing artifacts in the central Maven repository.
      There is a lot to say to Java build & dependency tools. But the article is already long enough and I just gave a very short intro to this tools. I hope it was anyway good brain food 🙂

      Reply

  20. One thing I am missing from maven pom parsing is a list of the included plugins. While those are not libraries in the strictest sense of the word, nevertheless they influence the build result of your software and therefore deserve to be tracked as well.

    Reply

  21. I would love if the maven pom parser could extract maven plugins from the poms as well. While not libraries in the strictest sense of the word, plugins influence the result of your build in a major way as well, so I feel they deserve tracking too.

    What I really don’t know is how well something like https://github.com/apache/maven-plugins or git://git.apache.org/maven-plugins.git can be tracked by versioneye, though.

    Reply

      1. True, there I can follow the plugins, but if I go to one of my projects and look at the dependencies, it would be nice if the plugins could be listed there in addition to the libraries. Would make the “Follow all” button you recently announced much more useful 🙂

    1. Totally agree on that. I guess I will write a separate blog post about bower with more details. Bower is currently really not perfect. But it’s a very young project. There is a lot of room for improvements and I hope they will come soon.

      Reply

  24. Composer/PHP does not support multithreading or multiprocess downloads due to no native support in the PHP language itself. This is a huge issue (con) when you have many package dependencies in your project, it can take literally forever to download. I’ve also found many inconsistent issues when trying to update projects by having to manually clear composer caches with rm -rf etc.

    The no signature thing is major. With so many projects and developers trusting these repositories they need to get their act together and quickly.

    Other than signatures, what are these package manager repositories doing in terms of mitigating actual malicious packages themselves? Are they performing any type of code analysis? I highly doubt it. Everyone’s blindly trusting the other which can be extremely dangerous.

    Basically no mater what package manager you use, do not use it on your production systems, in any circumstance. Other than high probability of stuff breaking, someone needs to actually look at what the hell it has pulled down in to your codebase to be pushed up…

    Reply

    1. Code analyses is not the job of a package manager.
      If you don’t use any package manager how do you deal with open source libraries? Do you download them with your browser? How is that better? Or do you don’t use open source libraries at all?

      Reply

      1. I’m not advocating to not use a package manager. For a developer, they are fantastic. I’m saying that it’s a common threat between all languages that security seems to be last on the list. This is unfortunate. I could argue that Google isn’t an antivirus company yet they provide Google Safe Browsing project to keep people away from unsafe websites. Others have even used this in their products (multiple browsers, etc). http://en.wikipedia.org/wiki/Google_Safe_Browsing

        Attacks are getting more sophisticated and if someone managed to add a backdoor to one of the top projects to any of the mentioned language package managers, the damage would be massive.

        App stores also verify the integrity of applications to keep users safe. Personally I see package managers as being no different.

        I think this type of thing will come as more people become aware and require more security.

      2. OK. Now I get your point. And it’s totally valid!
        But I didn’t missed that. Security is part of this comparison. In the infographic there is a column “signed”. If there is a blue dot in that column then the packages on the server are cryptographically signed and with the public key of the author you can check that the package was not manipulated. You still have to trust the author of the package, but at least you can be sure that the package was not modified by a 3rd party. Unfortunately Maven and Lein are the only 2 ones who are enforcing signing. Most of the other package managers have that feature, too, but they don’t enforce it.

  25. Day to day at work I mostly use Cocoapods which I am very happy with. I had to unfortunatly spend some time getting to know Maven. I liked your review because it made me not feel stupid about feeling overwhelmed and confused by it. But I think Go might potentially have the best system, since they have integrated it very well with the language. A source code file just link directly to github for libraries it depends on. So essentially a go project does not need any configuration files either for building or package management. It still has a few rough edges but this seems to be the way to go. It helps that they have thought about this from the very start.

    Reply

    1. No! You are not stupid 😉

      I still have to check out go. People told me already about that feature. But how does that work with versioning? Do you always point to the master head? That can be dangerous, specially if the code is under active development. I guess you can also point to tags of a git repo. Right?

      Reply

      1. Last time I checked it was like that. Hence my “a few rough edges”. So I think people often clone repos today to keep better control. Apparently the (current) “go way” is to create a new repo path when your library is no longer compatible.

        You can fetch repos directly with git if you want a specific version though. But I agree this is a little primitive compared to having say a Cocoa Pods file specifying all the versions you are using.

        Btw I also think the Julia package system would be worth checking out. I program in Julia at the moment, but I can’t say I have quite understood the package system yet. But what I find intersting there is that you typically don’t use an external tool to fetch packages. You just add packages from the Julia command line (REPL). Like Cocoapods and Go they also build their package system on top of git.

      2. I’m still not sure if I like package managers based on git. The disadvantage is that the package manager is not anymore in control of the artifacts. The owner of the repository can every time delete a tag. And the tags are not signed.

  27. It appears nobody has mentioned Cabal yet, which is the corresponding tool for Haskell. You might want to include it if and when you review a larger set.

    Reply
  28. Pingback: Comparison of Application Level Package Managers | Robert Reiz

  30. You’re mixing up objectively beneficial features like package signing with subjective crap like “JSON-based”. Fail.

    Reply

    1. It’s not easy to pick the right columns. Everybody wants to see something else. But obviously JSON is hip nowadays and XML was hip 10 years ago. The column is just a result of Zeitgeist.
      But which other feature would like to see instead of JSON?

      Reply

  31. Nice writeup. I use Maven mostly and I have to agree with all the cons you’ve listed. I’ve used Ivy on top of it sometimes and it is a bit nicer to manage but it only helps with some of the problems.

    I’ve used Dart’s Pub manager as well and I really like the approach.It’s still young and missing some features but compared to Maven it’s heaven. Would be interested to see it included in the next write-up.

    Reply

    1. Thanks for the Feedback 🙂
      I can not promise you that Pub will be included in the next comparison. First of all we have to take care of Nuget and CPAN. But Pub is on my radar 😉

      Reply

  32. When you write your follow-up article, you might find this comparison table I made useful: https://docs.google.com/spreadsheet/lv?key=0AnVCQSzTU_medDBUWTM2cGJodkYwUXdPZ3NJT3hycFE&usp=drive_web

    And I second the calls above from Perl hackers to talk about testing infrastructure. CPAN Testers (a crowdsourced run-your-tests-on-platforms-you’ve-never-heard-of system, see http://static.cpantesters.org/) is a thing of wonder, and I don’t believe any other language’s packaging system has anything like it – but they should!

    Reply

  33. Your post is so informative and you are sharing very useful topic. I am so inspired to read this post and I have learned many new things in your post which is informative and useful for me. I am also thankful to you for sharing this informative post with us.

    Reply

  36. Think of the Maven groupId as a namespace. Many package management systems, which don’t support namespaces, will end up with similar packages having some really long, convoluted, and or confusing package names to avoid namespace collisions. CPAN, among others, suffers from this and devotes entire web pages of advice to module authors on how best to avoid this kind of problem. Giving each author a namespace(by convention, the namespace is usually derived from the author’s domain name to ensure uniqueness) avoids all of these problems allows the packages to be named sensibly without fear of namespace collisions.

    Reply

    1. I don’t think that’s true. As you can see here https://www.versioneye.com/statistics there are almost as many Ruby and Node.JS projects out there as Java projects. And the Ruby names are all quiet short and meaningful. Name collisions are not a big deal.
      Beside that the groupId is not just a name space. It is the reverted domain of the owner/company. And many groupIds are far too complicated and far too long. For example:

      groupId: org.codehaus.jackson
      artifactId: jackson-mapper-lgpl

      The name “jackson” is redundant in this example. In my opinion the gorupId here should be simply “org.codehaus”. But even “org.codehaus” is unnecessary long. Why not simply “codehaus”?

      The best naming convention has GitHub. It’s dead simple “owner” / “project”. For example “codehaus/jackson-mapper-lgpl”. This naming convention works for 50 million projects now and the names are still shorter than in the Maven ecosystem.

      The groupIds and artifactIds in the Java world tend to be very long. In my opinion that reflects that Java developers tend to over engineer. And don’t get me wrong, I’m writing Java code every week by myself.

      Reply

      1. Namespace collisions are never an issue… Until they are. 😉

        Whether or not namespaces are actually necessary and establishing the intended function of the Maven groupId are separate issues. You can assert that the groupId isn’t necessary, if you wish, but, even if you are right, that doesn’t necessarily mean that it wasn’t intended to function as a namespace.

        The groupId being an inverted domain name is a convention not a technical requirement and it stems from the suggested Java package naming conventions which, incidentally, pre-date Maven. To quote the Java language specification, “The suggested convention for generating unique package names is merely a way to piggyback a package naming convention on top of an existing, widely known unique name registry instead of having to create a separate registry for package names.” Java piggy backed onto an existing naming convention for its package names and then Maven piggy backed onto the existing Java naming convention. I’m not saying its the best convention in the world. I’m just telling you where it came from.

        There are plenty of Maven artifacts out there which do not adhere to this convention just as there are plenty of Java classes out there which do not adhere to this convention. For example, commons-logging:commons-logging:1.2 doesn’t use an inverted domain name. If you want to use the “owner” / “project” convention with Maven then there’s nothing technically stopping you from doing so. Your groupId can be anything you want just as your package names can be anything you want. I don’t know, offhand, if the Maven repository servers enforce this convention when you’re attempting to register a domain name or not. If they are, then its a procedural requirement that should be attributed to the maintainer of the repository and not Maven itself. I’ve created many Maven artifacts and deployed them to our local Nexus repository without ever proving that I controlled to the domain name indicated by my groupId.

        From where I’m sitting, you’re just complaining that the author(s) of Jackson are following conventions that you don’t like and or are bad at naming their artifacts. 😛

        Furthermore, in the context of Maven and Maven artifact repositories, the groupId can also be used to uniquely identify the organization which created the artifact. Sure, the repository can track this information outside of the artifact metadata based on who uploaded it but I would argue that it is helpful to have it in the metadata directly. Take SLF4J, for example. The artifact indicated by the coordinates org.slf4j:slf4j-api:1.7.9 is effectively abstract and doesn’t do anything unless you also include a logging implementation on your classpath. However, what if you don’t remember the artifactId that you want or don’t know what implementations are available? If you don’t have a groupId, you then have to go to the repository server, look up the artifact, find the “author” of the package, and then look for other packages that they have published. Or, in the case of Maven, you can just copy the groupId out of your POM and search for it in one of the repository servers. So, in a matter of seconds, I was able to take the groupId org.slf4j and figure out what logging implementations are available: http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.slf4j%22

        In my opinion, the Maven groupId, while arguably more verbose, is not without its advantages.

      2. “Namespace collisions are never an issue… Until they are.” Good point 🙂 One for sure. In the Maven ecosystem naming collisions are pretty unlikely 🙂

        “From where I’m sitting, you’re just complaining that the author(s) of Jackson are following conventions that you don’t like”. Yes! That’s true 🙂 I like short names, but in the Java world everything is long. The imports are long, groupIds are long and sometimes even artifactIds are long.

        You are right with many points, but regardless of that, Maven is the only package manager there you need 3 coordinates to identify a package. All other package managers I know need only 2 cooridnates! A unique name and version is enough.

        The Clojure guys are using Leiningen as package manager with a regular Maven backend. At clojars.org groupIds and artifactIds are exactly the same. So pretty much they skipped the groupId, but because they rely on Maven they fill the groupId with the artifactId. That works pretty good for them.

        But however. It is how it is. And as Java devs we have to life with groupIds and artifactIds. If we like it or not 🙂

  37. “Dependencies are checked, fetched, resolved and installed on each compile time.”

    This is technically true(depending on where you think the dependency is being “fetched” from) but it implies that dependencies are downloaded for each build(which is not true at all). There are can potentially be many levels of caching but, even out of the box, there is always at least one layer of caching. The default(out of the box) dependency resolution mechanism looks something like this,
    1) Maven will look for the requested dependency in $HOME/.m2/repository and use it if it is found.
    2) If the dependency could not be found locally, Maven will attempt to download it from Maven central.
    3) Once downloaded, all subsequent builds will use the dependency from the local copy without downloading it again.

    Additionally, $HOME/.m2/settings.xml can also be configured to download the artifact from a mirror repository instead of from Maven central. This is actually the configuration which is preferred by the maintainers of Maven central because it drastically reduces the load on their servers, particularly when a organization(like mine) has its own repository. This is also common because it makes it much simpler for developers in an organization to share artifacts which are not open source but which do need to be shared within the organization. So, with our configuration, the process looks like this,
    1) Maven will look for the requested dependency in $HOME/.m2/repository and use it if it is found.
    2) If the dependency could not be found locally, Maven will attempt to download it from our mirror repository.
    3) If our mirror repository cannot find the dependency, it will attempt to download it from Maven central, store a copy of it in its mirror, and transparently relay the artifact to the build.
    4) All subsequent builds which need to download this dependency will then obtain it from our mirror and the mirror will not contact Maven central.
    5) Once the dependency has been obtained, all subsequent builds will use the copy of the artifact from $HOME/.m2/repository without contacting our mirror repository.

    The idea behind the cache in our home directory is to avoid downloading dependencies more than once. Having an organizational repository takes this a step farther and ensures that our organization will only download each dependency once as opposed to each developer in our organization only downloading each dependency once(which is what would happen without an organizational mirror). This holds true for building multiple projects as well. For example, if we have 5 difference projects which need the dependency org.slf4j:slf4j-api:1.7.9 then that dependency will get mirrored into our $HOME/.m2/repository folder the first time a build needs it and never again regardless of which project asked for it. This does not hold true for some other module systems that I have seen(like NPM) which seems to want to install dependencies directly into the current directory(once per project).

    Reply

    2. Everything you wrote is true and correct. I know pretty good how the dependency mechanism of Maven works. I worked for many years in Enterprise environments with the same setup. We used mostly Nexus as internal repository server. But you misunderstood my sentence or I was not clear enough 🙂

      That I wanted to point out is that Mavens life-cycle is not always an advantage. The different phases depend on each other and if I call “compile” it will automatically check my dependencies. Somehow that makes sense but many times not. Sometimes I just call a goal to generate docs for example and then it still is checking my dependencies and telling me that it can’t perform that gaol because some el-api can not be fetched. And that missing dependency has nothing to do with my goal I just called. That is specially annoying if you are traveling in a train and you are offline and forgot to run “mvn compile” at home before you entered the train 🙂

      In Maven everything is tied together. I mean the pom.xml is a huge Moloch. It’s not just the place where you describe your dependencies! It’s also the place there you describe all kind of other things, like profiles, documentation, executing tests, javadoc generation and so on. In other languages these kind of things are more decoupled from each other. In the Ruby world for example you describe your dependencies in the Gemfile. And there is nothing else in the Gemfile beside the dependencies! For configuring/generating docs you use another tool. For configuring & executing your tests you use again another tool. So you have very small and specialized tools following the Unix philosophy. And because you have very small and specialized tools for each job, its very easy to replace them. I like that approach much more.

      Reply

      1. What you are describing is as a “better” approach sounds disturbingly like the old free-for-all Ant builds that Maven replaced. With Ant, you define exactly how and when you want everything done. I’ll grant you that, in theory, this is superior. In practice, however, I find that it’s more trouble than it is worth. When you find yourself moving between projects a lot, you immediately notice that no two Ant builds are the same. Each one of them is a work of “art” and some of them don’t work on multiple operating systems or even other computers if their author was inconsiderate enough to use OS specific file paths, absolute file names, etc. Ant builds are often so complicated that it can take a significant amount of time just to figure out how to use the build. I can’t even recall how many times I’ve picked up Java projects that I couldn’t figure out how to compile because the Ant build was the logical equivalent of a tangled pile wires.

        The other thing to keep in mind is that Maven was designed, from the ground up, to do more than just dependency management. It’s not that it was designed for dependency management and then had a bunch of bloat added in later. It’s designed to do all of those things using a convention-over-configuration philosophy. It’s not nor was it ever intended to be simply a package manager. It’s a build tool which was designed based on lessons learned from Ant. So, yes, a build tool is going to do a lot more than just basic dependency management.

        I can see how your train scenario might be annoying but I think the advantages of Maven’s build life cycle far outweigh the disadvantages. When you are using a continuous integration server to automatically deploy your code to a quality assurance environment, then there are certain tasks that you always want to happen before the code is deployed. IE: I always want my code to be compiled, I always want my tests to be run, I always want my documentation to be generated, I always want my documentation to be deployed and I only want my code to be deployed if all of those things complete successfully. There’s no sense in deploying the code at all if there’s a failing test because we already know that something is wrong. Admittedly, you can do all of these things with what you describe as a “decoupled” approach. But, then I could say the same thing about the shell scripts I have in Jenkins to perform all of those tasks against Perl code. If you only have one project to deal with, this might not seem like a big deal. However, if you have dozens of them then I guarantee you that their builds will diverge over time and function very differently. This even comes into play when you on-board new developers or try to bring in outside contractors. If a new developer comes in, I want them to know how be able to check out the source and compile it within a few minutes without having to read a wiki page(which is likely out of date) about how to build the project and then ask other developers questions because they keep getting weird errors. Maven’s conventions address these kinds of problems but, given the choice, I think your irritation on the train is the lesser of two evils.

      2. I worked with Ant for many year before I switched to Maven and Maven is an improvement! Ant is so worse because it has no “convention over configuration”. That’s the reason why every Ant project looks different. Maven brings a unified structure to the project, that was a clear improvement. I’m also aware of it that it was designed as build tool, not just a package manager. And because Java is not dynamic language the “Maven way” makes somehow sense. What is criticize is that it’s not decoupled. Maven would be already much nicer if it wouldn’t force us to write everything in 1 single huge XML file. Imagine a Maven there you have multiple config files. For example a “maven-dependencies.json” only containing your dependencies! And a “maven-plugins.json” only containing plugins. And a “maven-profiles.xml” for profiles. And you could perform goals independent from each other and even use different tools to do certain things.

        I have more than 1 Ruby project right now and the configuring a Continuous Integration Server for my project is not such a big deal. On CodeShip the build configuration looks pretty much like this:

        1. bundle install
        2. rspec

        As you can see I use bundle to install the dependencies and RSpec to run my tests. Somebody else might use another tool for the tests, maybe RUnit or even something I never heard of. That works pretty good 🙂

  38. “NPM supports semantic versioning and has an own operator which supports fetching always the newest patch/minor version of a package.”

    As far as I’m concerned, this should be a con not a pro.

    Builds need to succeed months or years after the code is written or the code has no value whatsoever. I can’t afford to have a developer check out our source code 18 months after the code was written and be unable to use it because the dependencies which he installed are newer than the dependencies which were installed at the time the code was written. When a build is run, it needs to download the same versions of your dependencies and(more importantly) it needs to download the same versions of your transient dependencies each and every build. This is something that Maven does extremely well and it is something which NPM is absolutely terrible at. This “feature” forces us to check the entire node_modules directory into subversion just so that we can guarantee that our builds will still work years from now, if our development focus shifts to other projects.

    PS: If I’m unfairly maligning NPM please tell me how to do things differently because I really hate having dependencies checked into our source control. I much prefer Maven’s approach whereby we only check the source and the POM into version control and the exact same dependencies are downloaded each and every build even if we run the build years later.

    Reply

    2. Just because NPM has semantic version feature it doesn’t mean that you have to use it 😉

      http://semver.org/ makes a lot of sense. Not just for NPM hipsters, it makes also a lot of sense for big Enterprises 😉 The tilde operator (~) is a nice thing. Assume you define “~1.2” in a package.json file. That will always download the newest patch version. For example “1.2.3”. And if you execute it 18 months later maybe it will download “1.2.8”. That must not be the newest version! Maybe the newest version is already “2.0.0”. But with the tilde operator we defined that we only want to fetch the newest patch version. And by definition patches are always backwards compatible. So that shouldn’t be a risk and it shouldn’t break your build 18 months later.

      But you don’t have to use the tilde operator in NPM. You can also just work with fixed version numbers if you like. In that case you will ALWAYS get the exact same dependencies. Even 18 years later. And there is really no reason to checkin NPM packages into Subversion or Git. Just put the NPM package directory on the subversion/git ignore list and work with fixed versions in the package.json file.

      Beside that Maven has also a version operator which allows you to define version ranges or to fetch always the newest version. But the version semantic in Maven doesn’t directly support semantic versioning 😉 And pretty much all new package managers support semver directly.

      NPM is much younger than Maven and the approach of NPM is totally different. They come from different backgrounds. Keep in mind that a Node.JS developer usually does not deploy his Node project on a Java Application Server. Same is true for Ruby, Python and many other languages. In Node.JS the “app server” is just another dependency in your package.json file. A Node.JS project is meant to run on his own. You copy the project directory to the server and start the application with a single command. The only requirement on the server is the Node.JS runtime. And the same is true for a Ruby, Python or PHP web application. You can take this idea one step further with Docker containers. You prepare a Docker container with the desired runtime and just copy your application into the container. You can copy the Docker container to any server and with “docker run CONTAINER_NAME” you run the application.
      That approach is completely different from Java Application Servers where you have shared resources. In my opinion Java App Servers are just not state of the art anymore. They have a lot of disadvantages and are quiet slow. I migrated all my projects into Docker containers and I will not go back to Java App Servers 🙂

      Reply

      1. “Just because NPM has semantic version feature it doesn’t mean that you have to use it”

        So, what if I don’t use semantic versioning directly in my project but my dependencies or my transitive dependencies do use it? If I don’t use the tilde in my project, will that also prevent it from coming into play if it was used in one of my dependencies or in one of my dependencies transient dependencies?

        “And by definition patches are always backwards compatible. So that shouldn’t be a risk and it shouldn’t break your build 18 months later.”
        That’s theoretically true but you know what happens when you assume… Not everyone has good versioning etiquette and I’ve seen these guidelines violated on many occasions. I’ve even had my code broken by it, from time to time. The water gets even muddier if your code depends on internal API functions, as a workaround for a functional deficiency in the dependency, and those internal API functions(which you’re not technically supposed to be using) change between versions. You can’t always afford to refactor code to support a newer dependency when you’re just trying to make a minor patch to your own code and the developer might not immediately realize what went wrong and waste a lot of time trying to figure it out, especially if they are not very familiar with the project or have spent a large amount of time away from it.

        On the flip side, it is also true that enterprise applications do struggle with keeping their dependencies up to date over time. However, this often has more to do with shifting development priorities than it does with a developer simply forgetting to check for when new dependency versions are available. IE: Semantic versioning won’t help you at all if the code hasn’t been rebuilt in 18-36 months because development priorities have shifted to a different project. In that scenario, I would prefer to be responsible for manually bumping dependency versions(when we eventually return to that project) than to return to the project, have it not build(or worse throw weird runtime errors), and to not be sure what changed.

        “Beside that Maven has also a version operator which allows you to define version ranges or to fetch always the newest version. But the version semantic in Maven doesn’t directly support semantic versioning ”
        Yes, and almost nobody uses it for many of the reasons I’ve mentioned above. 😛

        The app server being a dependency is a nice touch, I’ll give you that. Though, you can sort of do the same thing with tomcat and jetty maven plugins, for development purposes. I completely detest “enterprise” Java servers like JBoss because their class loaders function entirely backwards and ignore the dependencies in your WAR if the dependency is already available in the app server itself. IE: You have to install your dependencies on the server before your application will run. I wouldn’t mind if the dependency in my WAR would override the one on the server and the server’s version was used if I didn’t provide one… But that isn’t what it does. It makes me feel like I’m using CPAN all over again. I hate installing dependencies on a server. I want them deployed with my application, which is exactly what a WAR does if you use a plain servlet container. I generally stick to servlet containers like Tomcat for this reason. With that said, the biggest problem with Java servers isn’t their performance. You’ve been able to get acceptable performance from modern servlet containers ever since support for NIO was added. The real problem with Java applications is the maximum heap size. The JVM was designed in the 90’s when RAM was a scarce commodity so, to ensure that multiple applications could coexist on a single server without tripping on each other, they built in the max heap size so that you could allocate the server’s memory on a per application basis in advance. Unfortunately, what this means in the here and now is that most Java applications use more memory than they actually need and use memory all the time as opposed to when they are being accessed. This is why Java hosting is universally more expensive than PHP hosting.

        While it seems a bit off topic, the bottom line is that there is no one language to rule them all. The “best” language is going to vary based on the task at hand. That being the case, there are still times where I choose to use Java simply because it is a statically typed language. Sadly, I have yet to see another platform which can match Java for static typing(and consequently IDE support). If I need to build something that works every time without the possibility of error, I have a much easier time delivering on that requirement in Java than I do in other platforms because the possibility of dynamic runtime behaviors are far far lower.

        I’ve looked at Docker briefly and it looks like it would be, at a minimum, very very handy for simplifying the deployment our old legacy Perl code.

      2. “If I don’t use the tilde in my project, will that also prevent it from coming into play if it was used in one of my dependencies or in one of my dependencies transient dependencies?” Very good question! To be honest I didn’t though on that. I would say no! It doesn’t prevent! From that point of view it makes sense that you checkin the dependencies into Subversion. Did you checkout NPM Enterprise? https://www.npmjs.com/enterprise? I never used it but maybe it has a solution for this kind of problems.

        It happened to me already as well that a patch/minor version broke my build. The Maven project introduces a not backwards compatible change into their APIs with version 3.2.0. And it happened to me at some RubyGems projects as well. But it’s not the rule. In my experience it’s really a rare exception and I still believe and rely on semver.

        I agree with you that library projects shouldn’t use something like the tilde operator at all. In a library project all dependencies should be fixed! Simply for the reasons you named already.

        Docker is not just nice for deployment. It’s awesome because each artifact is “executable” and it contains ALL dependencies of the project. Not just the library dependencies. It also contains the runtime, compiler and so on. If you want to ensure that your artifact can be executed in 18 years, then a Docker container is probably the best thing you can do. Beside a full virtual machine 😉

        Another big problem of a Java Application Server Infrastructure is that the Software provider which provides the WAR or EAR replies on the Java App Server and his shared resources. Means that Software provider has to deliver something which runs on the JVM. If you have Docker infrastructure the Software provider has more freedoms. The Software provider can pick any language and any compiler/runtime and dependencies. He just have to provide an executable Docker container. But I guess that is another blog post I should write about 🙂

      3. I’ve never heard of NPM Enterprise before but it looks functionally similar to Nexus. I’ll have to read up on it. We really only use NPM for building our web front ends. Unless someone can give me a compelling reason to do so, I have never been able to find a good reason to write server side code in Javascript. In many ways, even PHP is too dynamic for my taste and(unlike javascript) it has first class support for object oriented programming. It’s really hard to go back to what feel like flat text editors after you’ve spent years being spoiled by IntelliJ’s Java support.

        “If you have Docker infrastructure the Software provider has more freedoms. The Software provider can pick any language and any compiler/runtime and dependencies.”
        Well, you can use Docker to deploy Java applications too, including the JVM and the application server. But, even if you do that, you still have the heap size problem that I mentioned before so your hosting costs are still going to be higher than a docker app written in something like PHP or javascript.

        Docker would be a great thing to blog about, from my perspective. It’s something that I’ve read about but have very little experience with. 🙂

      4. Hey John. I thought again about your transitive NPM dependency problem. That’s really an issue. In Ruby this problem is solved with a lock file. You define your dependencies in a Gemfile and the very first time you execute “bundle install” to actually fetch and resolve your dependencies the package manager will generate a Gemfile.lock, which includes ALL transitive dependencies with fixed version numbers. The Gemfile.lock is something you checkin into Subversion/Git. And if another developer executes “bundle install” 18 months later he will get the exact same transitive dependencies, because bundle will fetch only the versions which are defined in the Gemfile.lock. Unfortunately NPM doesn’t generate a package.lock file :-/ But I found this blog post: https://www.npmjs.com/package/lock-versions. If that’s not a solution you should check out the filters for NPM Enterprise.

        IntelliJ is pretty awesome. I use it since many years for my Java development. But for web applications I prefer Ruby, because the development process is so much faster than with Java. And beside that I don’t know 1 single good web framework in Java. I worked for many years with Struts and it sucks! Also worked for 3 years with JSF and that one was even worse! 2011 I finally learned Ruby on Rails and that was kind of awesome. Somehow everything made sense and since then I use it for everything web related.

        Sure you can use Docker to deploy Java applications as well. And I hope nobody is so mad to install a WebSphere into a Docker container to deliver a couple HTML pages 😀

      5. I guess I’m stuck committing my node_modules folder, for the time being. 😦

        I’m of the opinion that server side web frameworks, in general, are obsolete. So, we don’t use web frameworks at all in any language. Everything we do is SOFEA(Service Oriented Front End Architecture). Thus far, I’ve successfully implemented this architecture in PHP, Perl, and Java. I firmly believe that it is a pattern that can be implemented in any language. Though, it will be easier in some languages than others. For example, In Perl I had to write my own rest framework(including my own url routing) because the options available on CPAN were completely inadequate. The only thing we build on the server are restful web services which consume/produce JSON and the web front end is responsible for doing everything else. We then layer Swagger-UI on top of the server and the web front end developers can interact with Swagger and build their UI’s without ever looking at the server code or the database. In Java, the only server side “framework” that I would consider using would be JAX-RS coupled with Google Guice(JSR-330) dependency injection.

        By choice, I would never use WebSphere for anything. It’s the same kind of crazy as JBoss and WebLogic, as far as I’m concerned. 😛

  39. Pingback: 9 different package managers | Smash Company
  40. Pingback: Mengenal package manager
  41. Pingback: Guideline für Open Source Software | SUCKUP.de
  44. Pingback: 1 – Which programming language has the best package manager?

Leave a reply to Ruby Programming Language Cancel reply