Improved license recognition

VersionEye is monitoring now more than 1.2 Million open source projects and collecting all kind of meta information to this projects. One kind of the meta information is the corresponding license. Currently the VersionEye database contains licenses to more than 8 Million artefacts.

However, sometimes the maintainers of a project didn’t provide the license information that way that it’s easy to read and recognise for machines. That is specially the case for Python and the .NET platform. Take for example the gpkit Python library. In the license field they provided the full license text, not just the license name.

python-gpkit-00

That doesn’t match very well with the license whitelist in VersionEye 😉 That’s why we improved it now!

All together we have 11335 Python licenses like that in our database and they belong to 1989 unique projects on PIP. Our new license crawler could match 9933 licenses to SPDX identifiers. That are 1799 unique projects on PIP we could assign a SPDX identifier to, which didn’t had one before. For more than 90% of this projects we could recognise and identify an SPDX identifier. And now the same library on VersionEye looks like that:

python-gpkit-01

You see clear that it’s MIT license and now this works well together with our license whitelist 🙂

In Nuget, the package manager for the .NET platform, many license names look like this.

csharp-00.png

“Nuget unknown”. That is the case if the maintainers provided a license link but didn’t provide a license name. Our new license crawler is now following this links and with machine learning it tries to identify a known license. If the similarity to a known license text is higher that 90% we assign the corresponding SPDX identifier to the software library in our database. And now the same package looks like this and you can see immediately that it’s the MIT license!

csharp-01.png

That also helps to use these packages together with our license whitelist. For the .NET packages our recognition system was not quiet as good as for Python. For .NET we could only identify 65% of the licenses of packages with a link but without a license name. Stil not bad and much better than before 😉

This are just the first results. Of course this is also work in progress and the recognition will become better in future.

Your feedback is very welcome.

Suggest new licenses

VersionEye is monitoring now more than 1.2 Million open source projects and collecting all kind of meta information to this projects. One kind of the meta information is the corresponding license. Currently the VersionEye database contains licenses to more than 8 Million artefacts.

However, it is not always possible to fetch the license automatically. Sometimes things go wrong and sometimes the license is not available through a repository API. Sometimes human interaction is required to find the license for an artefact. Now everybody from the VersionEye community can suggest a license to an artefact. If you are on a VersionEye product page with an unknown license you will see a new “Suggest a license” link now.

screen-shot-2017-01-18-at-19-59-14

By clicking on the link you will come to a new page where you can suggest a license for the corresponding artefact and the form is already pre filled for you.

screen-shot-2017-01-18-at-20-01-36

By submitting the form above the VersionEye team will receive an email notification with the new license suggestion. After the submission was reviewed and approved the license will show up on the page.

I hope that many of you will use this new feature! 🙂

New icons for visibility scope and private projects

Up to now it was not obvious which VersionEye project is a “private” project. That means a project from a private GitHub or Bitbucket repository. Projects created through the VersionEye API are also considered as private projects. To monitor private projects you need to have a paid subscription. To make the private projects more visible we marked them with a lock icon in the project overview table!

Screen Shot 2017-01-12 at 14.38.59.png

In the example above the first 2 projects are private projects, they are created through the public VersionEye API and that’s why they are marked with the lock icon.

Don’t confuse “private” projects with the visibility scope. By default every project is publicly visible to everybody who knows the project URL. That makes sharing information very easy. But don’t worry your projects are NOT part of the search index, that means it will not pop up in search results! In the project settings you can limit the visibility scope to collaborators only.

Screen Shot 2017-01-12 at 14.26.58.png

That means that your project is only visible to members of your organisation. So if somebody knows the project URL and he/she is not member of your organisation he/she will not be able to see the project report! The projects which have limited their visibility are marked in the project overview with an broken eye, like the last 2 projects in the first image.

VersionEye Maven Plugin 3.11.1

Version 3.11.1 of the VersionEye Maven Plugin is out and here are the change logs. This release brings a couple improvements and BugFix!

BugFix

Assume you have a Maven multi module project with 3 modules. Now you add the VersionEye Maven Plugin to the parent pom and run mvn versioneye:create. That will create a new project with subprojects in VersionEye. Now on your CI system on each build you run maybe this goal: mvn versioneye:securityAndLicenseCheck. That will always update the VersionEye project with the current dependencies from your Maven project and check them for security vulnerabilities and license violations. Everything works so far.

Now you are adding a new Maven module to your Maven reactor build and the next execution of mvn versioneye:securityAndLicenseCheck will fail, because it will try to update an non existing module on the VersionEye server.

This bug is fixed now! Version 3.11.1 of the VersionEye Maven Plugin can handle newly added Maven modules. If a new module is added it will automatically create a new subproject for that on the VersionEye server.

Improved Merging

For Maven multi module projects one project is created for the parent pom and for each module a project is created on the VersionEye server and merged into the project for the parent pom. This merging process was working through the GAV coordinates of the parent pom. That was leading to problems if the same project was monitored twice on a VersionEye server. The new merge mechanism is independent from the GAV coordinates, it only relies on the project ID from the VersionEye API.

Support for Project License

The VersionEye Maven Plugin is resolving all dependencies locally and only sends a pom.json file to the VersionEye API, which contains only the needed information. Now this pom.json file also contains the license from the pom.xml file and the project license is displayed in the web interface.

Screen Shot 2017-01-12 at 13.31.55.png

 

Refactored Notifications

VersionEye is monitoring more than 1.2 Million open source projects by now. You can follow any of those OS projects at VersionEye and as soon a new Version of the followed package is release you will receive an email notification.

Facebook like notifications

The notifications which are created and send out to you are available in the web interface as well, but it was a bit hidden up to now. In the current version new notifications count is displayed with a red banner in the main menu. Just like you know it from Facebook and other social networks. Here an example:

screen-shot-2017-01-05-at-14-00-00

Directly next to “Organisations” in the main menu you can see a red 1. That means you have 1 unread notification. If you click on it you get to the notification view and the red notification count disappears.

screen-shot-2017-01-05-at-14-00-53

That way you see immediately if there are new releases out there. Even before you get the email 😉

Notifications via API

The notifications for new artefacts you are following are available via the public VersionEye API as well. It works exact the same way. If you have unread notifications the API will return a list of notification entities and each of the unread notifications will have the flag “read = false”. If you fetch the same resources again the “read” attribute will be on “true”.

screen-shot-2017-01-11-at-15-03-47

Beside that the API also returns the information if the notification was already send out vial email or not.

By the way. There is a new VersionEye – Slack connector which is already using this API Endpoint. With that you can receive the notifications directly into your Slack channel!

Leave a comment here if you have feedback.

New Team Notifications

VersionEye has already a concept for organisations and teams. Up to know it was possible to assign 1 team to multiple projects, but a project was always connected to exactly 1 team. The team members received All notifications in the period which was selected on the project. That was not ideal because specially in big organisations there are different groups of interests. Some people are only interested in  security notifications, some only in license notifications and others are maybe interested in version and security notifications. Beside that different groups of people want to get their notifications on different week days. Now all this possible!

Now the notification and period settings are not on the project level anymore. There are on the team level now. Each team can choose which notifications there are interested in and on which week days they want to receive the email notifications to that.

Maybe the developer team is interested in outdated versions and security vulnerabilities and they want to receive email notifications from Monday to Friday but not on the weekend. That would look like this:

team_notifications_01

The team “security_officers” is only interested in security notifications on every day of the week. That would look like this:

team_notifications_02

And now a project can be assigned to multiple teams. In the project settings tab, at the bottom, multiple teams can be selected which should be assigned to the project.

team_notifications_03

In the example above 2 teams are assigned to this project. Each team will receive email notifications for different aspects of the project, on different days of the week.

This new model offers much more flexibility and is a really good fit for big organisations with many teams and many projects.

Try it out and leave a comment here if you have feedback.

New Privacy Setting

If you have an account at VersionEye your username will show up in autocomplete fields. One of this autocomplete fields is for example in the teams view. If you want to add a new team member to your team you simply start typing their username and immediately you get usernames suggested like in this example:

screen-shot-2016-11-29-at-14-26-07

If you don’t want that your username shows up in the autocomplete fields, for privacy reasons, you can turn it off now. Simply go to your profile settings and then to “privacy”. Here you have an option “My profile shows up in autocomplete fields?” which you can turn off by selecting the value “No”.

Screen Shot 2016-11-29 at 14.26.30.png

After saving your new settings your username will not shop up anymore in autocomplete fields.

Screen Shot 2016-11-29 at 14.26.56.png

Try it out yourself and leave a comment if you like this feature.