Component Whitelist

At VersionEye you can setup a license whitelist and assign it to your projects. Simply put open source licenses on the license whitelist you want to allow in your projects/company. VersionEye will check all your dependencies against the license whitelist and notify you if there is a license violation. This feature is used frequently at VersionEye.

Today I want to introduce the component whitelist. It is an extension to the license whitelist. Software packages who are on the component whitelist are marked “green” in the license tab, even if they violate the license whitelist. There are a couple of use cases for the component whitelist.

Assume you are using an open source package which is licensed under the MIT license. With the next release the maintainer is changing the license from MIT to GPL-3.0. That means the newest version of the software library would probably violate your license whitelist. But maybe you buy a commercial license for the library, so that you are allowed to use it in closed source software. In that case you could set that artefact on the component whitelist.

Here is an example. Assume you have a project with some dependencies which violate your license whitelist. Just like here:

component_whitelist_3

Now we will turn the 2 red dependencies green by putting them on a component whitelist. Simply click on the button “Manage Component Whitelists”. You will be redirected to this view.

component_whitelist_1

The component whitelist works similar to the license whitelist. You can have as many component whitelists as you want. Create a new component whitelist by typing in a name and clicking on the button “Create New Whitelist”.

By clicking on the name of the component whitelist you come to a view where you can edit the elements on the list.

component_whitelist_2

An element on the component whitelist has in general this structure:

GROUP_ID : ARTIFACT_ID : VERSION

For example:

org.apache.httpcomponents:httpmime:4.5

The expression above would whitelist version 4.5 of httpmime. And it would whitelist only version 4.5. Version 4.4 or 4.6 are not whitelisted!

Now let’s say we want to whitelist all version of junit. That would look like this:

junit:junit

Or whitelist everything in the group “org.apache”.

org.apache

And don’t forget to whitelist mail.

javax.mail:

All right. Now go back to the project and select the new component whitelist and click the save button. Now we will get this view:

component_whitelist_4

Now everything is green. There are 2 dependencies which violate the license whitelist, but because they are on the component whitelist they are marked green anyway.

This feature is very new. Please test it with caution and give feedback. Either here in the comments or on Twitter.

VersionEye SBT Plugin

Probably you know already the VersionEye Maven Plugin. Now there is a similar plugin for Scala SBT as well. The VersionEye SBT Plugin is maintained by Mark Paluch. He basically implemented the whole SBT plugin in 1 single day, because he is awesome!

sbt

Currently the plugin is not pushed to Maven Central or JCenter. Not yet! If you want to try it out, please clone the repository with git, navigate into the directory and run this to install the plugin into your local repository.

sbt publish

After that you can use the plugin in your own SBT projects. Simply follow the instructions on the README page here.

The cool thing is, that the plugin has a licenseCheck goal. This

sbt versioneye:licenseCheck

will update an existing VersionEye project with your current dependencies and check all dependencies against a central license whitelist on the server. If one of your dependencies is violating the license whitelist on the server, the plugin will exit with an Exception. If you run that goal on a CI server it will break your build if there is a license violation. If you work in a commercial team and you want to ensure that nobody is accidentally introducing a copyleft license, like GPL for example, then that’s the way to go ;-)

Now references are sorted by references

At VersionEye you can take a look which other open source projects are using a certain open source library. We call that references. If for example log4j is using the JUnit library, then log4j is a reference for JUnit. On the detail pages of each software library you can see how many references a software library has. This is how it looks for JUnit.

Screen Shot 2015-08-24 at 10.38.35

JUnit has more than 55K references. That means more than 55K open source projects are using JUnit as a dependency.

By clicking on the number you get all of the references listed. That’s not new, but now the list of the references is sorted by their reference number. That means the open source library/reference which has the most references is on the top. This is how it look for the JUnit reference list:

Screen Shot 2015-08-24 at 10.35.11

The most popular open source library which is using JUnit as a dependency is slf4j-api. Because slf4j-api itself has more than 17K references from other open source libraries. In the list of the JUnit references it’s the open source library with the most references.

The references are available via the VersionEye API as well.

Caching Rails assets with NGinx

Nginx is a very popular high performance web server. It’s very easy to configure and many Ruby on Rails developers are using it as a load balancer for Rails applications. Usually the setup looks like this:

Screen Shot 2015-08-18 at 10.13.19

Typically the rails application is running on Mongrel, Unicorn or Puma. These ruby application servers can deliver static assets (.png, .jpg, .js, .css) as well, but there are not really optimised for that. Nginx is much faster in delivering static assets like images, javascript and css files. Ideally your ruby application server should only handle requests to ruby controllers and generate dynamic content.

This is the Nginx configuration for load balancing between 2 ruby application servers, or any other application servers:

 
upstream unicorns {
  server 192.168.1.11:8080;
  server 192.168.1.12:8080;
}
 
server {
  listen 80;
  location / {
        proxy_redirect off;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_pass http://unicorns;
    }
}

At Rails all static assets are delivered under the route “/assets/”. Now we will tell Nginx to cache files from “/assets/” for 120 minutes. For that we need this 2 lines at the beginning of our nginx.conf file.

proxy_cache_path /tmp/nginx levels=1:2 keys_zone=veye_zone:10m inactive=120m;
proxy_cache_key "$scheme$request_method$host$request_uri";

The first line defines the directory where the files for the cached asserts should be stored. In this case it would be “/tmp/nginx”. We define the name space “veye_zone” with 10 MB for this cache. The 2nd line defines the structure of the cache key, which has to be unique.

Now we can add another location for “/assets” to our configuration.

server {
  listen 80;

  location /assets/ {
        proxy_redirect off;
        proxy_pass_header Cookie;
        proxy_ignore_headers Set-Cookie;
        proxy_hide_header Set-Cookie;
        
        proxy_cache veye_zone;
        proxy_cache_valid 200 302  120m;
        proxy_cache_valid 404      1m;

        proxy_pass http://unicorns;
  }

  location / {
        proxy_redirect off;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_pass http://unicorns;
    }
}

The 3 most important lines for the caching are these here:

        proxy_cache veye_zone;
        proxy_cache_valid 200 302  120m;
        proxy_cache_valid 404      1m;

They reference to the “veye_zone” cache, which we defined in the top of our nginx.conf. The 2nd line tells Nginx to cache everything with a response code of 200 and 302 for 120 minutes. And everything with a response code of 404 for 1 minute.

All HTTP requests to “/assets/” first will go to one of the ruby application servers, but the responses will be cached at Nginx for 120 minutes. All further requests to “/assets/” will not hit the ruby app servers, because Nginx can deliver the content from the cache.

The beauty at this setup is that it’s seamless to the deployment of the ruby application. The deployment process doesn’t have to copy the assets files to the Nginx server. Nginx is just automatically caching all assets.

VersionEye SBT Plugin

The VersionEye API is becoming more and more popular. And there are more and more tools which integrate the VersionEye API. Specially in Java Enterprise the VersionEye Maven Plugin is very popular.  It can handle complex reactor builds with Maven. But the requirement to support SBT natively is growing as well. That’s why I would like to have an SBT Plugin for VersionEye similar to the VersionEye Maven Plugin.

sbt

The requirements for the plugin can be found here. I posted the link yesterday on Twitter and somebody from the community stepped up to work on this :-) The project will remain 100% open source under the MIT license. I expect to get the first results in the next weeks. More help is always welcome ;-)

Auditlogs for Projects

At VersionEye you can add collaborators to your project. Up to now the collaboration feature was only there to get the email notification about out-dated dependencies in the project. With the last update the collaborators have almost the same rights as the project owner. They can:

  • Rename the project
  • Invite other collaborators
  • Change ALL project settings in the settings tab.
  • Trigger the ‘Re Parse’ button.

They can not:

  • Remove a collaborator. Just themselves.
  • Change the license whitelist of the project.

All this actions are logged in the Auditlogs. In the project settings tap at the bottom the Auditlogs of the project are listed.

Screen Shot 2015-08-14 at 14.13.45

That way everybody can do almost everything and the actions are documented and reproducible.

Project Settings Changes

The project setting at VersionEye just got an update. Up to now it was possible to select a secondary email address to which the project notifications should be send. This is removed now! It was a feature which less than 1% of all projects used. If only 0.83% of all projects are using a feature then it’s not worth maintaining the code for that.

Screen Shot 2015-08-13 at 16.39.28

Beside that the License Whitelist setting is removed from the settings area. Simply because it is already included in the license tab. No reason for redundancy.

The collaborators feature got an update as well. Now collaborators of a project can see ALL tabs and they have almost the same rights like the project owner. A collaborator can invite other collaborators. But he can not remove other collaborators from the project. That power has only the project owner and the admin (me). But he can remove himself from the project.

Screen Shot 2015-08-13 at 16.41.19

In the settings tab the collaborator only can set the period in which he would like to receive notifications about this project.

Screen Shot 2015-08-13 at 16.41.09

The other settings are only visible for the project owner and the admin. I’m not sure about this. Maybe all settings should be visible for all collaborators and then just log who changed what. What do you think? I’m looking forward for feedback.