Tag: Continuous Updating

Pull request support for GitHub

77% of all our signed up users are coming from GitHub and they use VersionEye to monitor their GitHub repositories. For a while now there is a VersionEye service hook in GitHub which calls back the VersionEye API on every git push event and updates the corresponding project in VersionEye with the latest dependencies from GitHub. But now you can use VersionEye to test your changes directly in a Pull Request and see the results of those changes in the GitHub Pull Request UI. This way you can even enforce rules that would prevent a Pull Request merge if a dependency is blacklisted or it is flagged with a security vulnerability.

Given that GitHub is moving away from Service Hooks towards native WebHooks, we changed our integration to do the same. If you are logged in to VersionEye with your GitHub account you can simply click the link “Create from GitHub” and navigate to a repository where you will see all supported project files and the corresponding branches. To monitor a project file simply flip the switch directly next to it.

Screen Shot 2016-08-16 at 14.05.48

In the example above the Gemfile in the develop branch is monitored by VersionEye and it works like this: Every time a new project file is imported from GitHub, VersionEye will automatically create a web hook on that GitHub repository, which will call back the VersionEye API on each “git push” event and if the monitored files are affected, VersionEye will re-parse the files and update the view in VersionEye immediately.

The cool thing is that this also works with pull requests! Assume somebody is forking your project, adding new dependencies and sends back a pull request. In that case the VersionEye web hook kicks in and checks if some of the dependencies have security vulnerabilities or unknown licenses.  For those cases, VersionEye will respond to the pull request with  a “failed” status message  and you can see the result directly on the GitHub Pull-Request tab, like this:

GitHub-pullrequest-VersionEye-check

By clicking on the “Details” link you can see a more specific report at VersionEye for that pull request.

gh-veye-02.png

A report like this is generated on every new commit which is added to the existing pull request. It’s similar to a build at TravisCI. Now you can still merge the pull request if you like but at least you are informed about security vulnerabilities and license issues.

In the VersionEye UI you will find a new sub menu “Pull requests” in the organisation view. Here are all commits from all pull requests are listed which belong to your organisation.

Screen Shot 2016-08-16 at 14.22.50

The interesting part here is how VersionEye is combining dynamic and static analysis. Your project is checked for security vulnerabilities and licenses on each commit and each pull request but also regularly independent from that. Even if there are no commits and no pull requests for a couple days, VersionEye will check your project once a day by default and send you an email if a new security vulnerability was found for one of your dependencies.

With this new feature the developers are getting instant feedback to their changes. In real time the dependencies are analysed and the developers informed. That saves a lot of Time & Money. We think it’s important to run this kind of analysis as early in the development process as possible. If your software is already bundled and published to a binary repository it’s too late for this kind of analysis because your software might be distributed already to a high number of devices/clients. Run your analysis as early and as often as you can 😉

And by the way, that’s all open source 😉 You can run the software on your own infrastructure and it works well together with GitHub Enterprise 🙂

This is a pretty new feature. If you find a bug or you have some kind of feedback, simply open a ticket here.

Now VersionEye has support for .NET

VersionEye had already support for 12 package managers and now we are adding Nuget, the package manager for the .NET platform from Microsoft! As Microsoft Windows is all about continuous updating, Microsoft will like VersionEye very much, because VersionEye is continuous updating for .NET packages 🙂

With fixed releases and snapshots together VersionEye is currently monitoring more than 70K open source .NET packages on Nuget. Here are the numbers from .NET / Nuget compared with package managers from other languages.

Screen Shot 2016-07-26 at 15.23.36

The above chart can be viewed on the VersionEye statistic page and is updated once a week. The numbers representing projects in official package mangers. Random OS projects on GitHub/Bitbucket are not taken into the calculation! Currently the C# .NET community has less open source packages than the Python, Ruby, PHP, Java or Node.JS community. As Microsoft changed their open source strategy a couple months ago, this might change in future. But this is the status quo!

Every .NET package has his own page at VersionEye, like this one for example.

Screen Shot 2016-07-26 at 14.20.57

Now you can follow any .NET package at VersionEye and you will receive an email notification as soon a new release for that package is available. But that’s not all. VersionEye is showing the most important KPIs to the package in the right upper corner. KPIs like average release time, external references, the license of the package and a dependency badge which can be integrated into README pages at GitHub, GitLab and Bitbucket.

VersionEye is also creating author profiles for the contributors of the open source libraries. By clicking on an author icon on a package detail page the author page shows up. Like this one for example.

Screen Shot 2016-07-26 at 14.48.58

On the author page you can see to which other .NET packages the author contributed to and what his/her keywords are. The keywords are clickable as well. This is for example the keyword page for JSON.

Screen Shot 2016-07-26 at 15.35.56

The keyword page can be filtered down by language and now C# is on the filter as well.

But the coolest feature is that VersionEye can monitor a Nuget project in your Git repository and notify you about out-dated dependencies and license violations. Now VersionEye can monitor project.json and *.nuspec files on GitHub, Bitbucket and Stash.

Screen Shot 2016-07-26 at 15.41.34

For monitoring one of this files, simply flip the switch in our Git integration view. The project report looks like that.

Screen Shot 2016-07-26 at 15.42.30

It’s that easy to get notified about out-dated dependencies and license violations 🙂 Projects like this one can be created and update via the VersionEye API as well. There already many native VersionEye plugins for build tools and package managers like Maven, SBT, Gradle and NPM which are using the VersionEye API. I’m looking forward to see some native VersionEye plugins for the .NET platform as well 🙂

TODOs

2 Things are still not finished for the .NET integration and are still ongoing.

  1. Release Dates: On some packages you will see “Release 116 years ago”. Of course that is wrong 🙂 On some packages the release date was not set correctly. Currently a background job is running which is fetching the release dates from an alternative source.
  2. License Names: The Nuget API doesn’t expose license names, only links to licenses. Currently we are working on it to resolve the links to meaning full names. For a couple thousand packages this is already done but for the majority this process is still ongoing.

Feedback

The .NET integration in VersionEye is pretty fresh. We rely on your feedback! Please test it and let us know if something is wrong. You can reach us on Twitter or simply open a ticket at GitHub. By the way. VersionEye itself is open source. Pull requests are welcome 😉

Security PDF

Maybe you know already the License PDF export feature at VersionEye. Now there is PDF export available for the security tab as well. You find the download link in the project detail view in the security tab.

Security-PDF

This PDF export contains a list of ALL dependencies from the project and sub projects with their security status and the date of export. Here you can see how the PDF looks like on my pet project.

Securiyt-PDF-Export

At the bottom of the document all vulnerable dependencies are listed with their security vulnerabilities and according links.

This is a very new feature. If you find a bug or you have an idea how to improve it, please let me know. Leave a comment here or contact VersionEye at Twitter.

PHP Runtime Compatability

Now VersionEye shows which PHP package is compatible to which PHP runtime.

PHP has different runtimes, just like Ruby and Java. There are different versions of the PHP interpreter, there the newest is currently the 7.X branch, which brings massive performance improvements. But there is also the HHVM runtime, developed by Facebook. HHVM is a virtual machine for PHP and Hack, which is using a JIT compilation approach. Not all PHP libraries are running on HHVM or on PHP 7.

The PHP-Eye project from Julius Beckmann is collecting travis-ci configurations from GitHub and the corresponding test results to detect if an PHP artefact is compatible to a specific runtime or not. This kind of information is not available for all PHP libraries. But for many of them. For the good ones who are doing continuous testing and continuous integration. For this projects a runtime badge can be generated. VersionEye displays for each PHP artefact 2 runtime badges at the top of the page.

VersionEye-PHP-Runtime

That way it’s easy to see on which runtime the selected artefact was tested an on which not. The runtime badges link directly to a page at PHP-Eye where the full compatibility matrix for the PHP package is displayed.

Screen Shot 2016-03-03 at 09.46.50

VersionEye is using this information for the notification emails as well. Now in the notification emails the tested runtimes are listed as well. Like in this example:

Screen Shot 2016-03-03 at 19.58.57

We hope this information is valuable for you and helps you to build better software. Your feedback is always appreciated 🙂

VersionEye Maven Plugin 3.9.1

VersionEye Maven Plugin 3.9.1 is out and it has a lot of new features!

Out-dated dependencies

Now the Mavne golas mvn versioneye:create and mvn versioneye:update output the out-dated dependencies directly in the console. Like in this example:

Screen Shot 2016-03-02 at 09.05.20

ignoreDependencyManagement

This new configuration option can be used to ignore dependencies which are defined in the dependencyManagement section of the pom.xml. By default the value of this switch is false. That means by default the dependencies which are defined in the dependencyManagement section of the pom.xml are processed and send to the server. If you don’t want that, just set the switch to false!

API Key

The way how the API key is fetched was completely refactored! Now it’s possible to keep the API key in the environment variable VERSIONEYE_API_KEY. That is specially useful for setups on a central build server like Jenkins for example!

The plugin will look for the API Key at this places in that order:

  • Environment variable VERSIONEYE_API_KEY
  • ~/.m2/versioneye.properties
  • src/qa/resources/versioneye.properties
  • src/main/resources/versioneye.properties
  • pom.xml file

That means if the environment variable VERSIONEYE_API_KEY is set, it can be overwritten by the~/.m2/versioneye.properties file. And that can be overwritten by the settings insrc/qa/resources/versioneye.properties and so on.

Proxy Settings

Until now the proxy settings could be configured directly in the pom.xml file in the plugin configuration part. Now the configuration for the proxy can be placed at ~/.m2/versioneye.properties as well. That way it’s centralised and not all projects need to configure it on their own. Just put the proxy settings into the ~/.m2/versioneye.properties file like this:

proxyHost=PROXY_HOST
proxyPort=PROXY_PORT
proxyUser=PROXY_USER
proxyPassword=PROXY_PASSWORD

If a project configures the proxy in the pom.xml in the configuration part of the plugin, that values are taken for that project. But otherwise the plugin will look for proxy settings at~/.m2/versioneye.properties.

mvn versioneye:delete

Now there is new goal for deleting a project from the server.

mvn versioneye:delete

The goal above will delete the project and all it’s modules from the VersionEye server and also remove alle versioneye.properties files from the source code. That is specially handy for big Java projects with many modules!

Improved Projects List!

In VersionEye there is “Projects” item in the main menu. Until now this view showed either your projects which are attached to your personal account or the projects from a selected organisation. But there was no view which showed ALL the projects you have access to, over all organisations/teams. That is improved now. Now by default you see ALL projects you have access to, separated by organisation/teams.

VersionEye-ProjectsList

The team names are clickable. That way it’s easy to navigate to the desired organisation and team, to get deeper insides. The old project filter is still there, that way it’s easy to filter down the project list!

 

Team Filter for Organisations

A new feature was released yesterday! The team filter for projects inside of organisations! That is specially useful if you have a lot of projects inside of an organisation and many teams. In big companies/organisations usually a developer is working full time on one team and maybe on different projects. Up to now the projects view inside of the organisation view in VersionEye listed by default ALL projects from the organisation. But if you are a developer in the “Backend” team you are not interested into all the other projects. You only want to see the projects which belong to your team! The “Backend” team! And exactly that is solved now!

VersionEyeTeamFilter

The projects inside an organisation can be filtered by teams now! If you belong to N teams, the last team is pre selected for you. That way by default you only see the projects which are relevant for you. Of course you can change the team filter afterwords to see projects from other teams or even ALL projects if you like.

Feedback welcome 🙂

VersionEye Maven Plugin 3.7.0

Release 3.7.0 of the VersionEye Maven Plugin is out and it brings a couple cool new features!

Support for Organisations & Teams

If a new project is created, by default, it is attached to the user account who created the project. In the Web interface the project can be moved into an organisation and then a team can be assigned to it. Until now that was a lot of manual work. Now with this release it can be automated. The new version of the VersionEye Maven Plugin has 2 more configuration options. Now the organisation and the team can be defined directly in the plugin configuration, like in this example:

<plugins>
    <plugin>
        <groupId>com.versioneye</groupId>
        <artifactId>versioneye-maven-plugin</artifactId>
        <version>3.7.0</version>
        <configuration>
            <apiKey>API_KEY</apiKey>
            <organisation>versioneye</organisation>
            <team>BackendDevs</team>
        </configuration>
    </plugin>
</plugins>

By creating the project with this command:

mvn versioneye:create

The project will be created at VersionEye and automatically assigned to the organisation, in this example to the organisation “versioneye”. And the team also gets assigned to the project, in the example above it would be “BackendDevs”. The organisation and team names are case sensitive! Please take care of that, otherwise it will not work!

This feature also assumes that the owner of the API key is part of that organisation and has the permissions to attach projects to that organisation and the permission to assign teams to projects! If that is not the case or the organisation doesn’t exist, the project still gets created, but then it belongs to the user who owns the API key. Just like the old behaviour!

securityCheck

Similar to the licenseCheck goal, there is a new goal “securityCheck”!

mvn versioneye:securityCheck

This goal checks how many of the project dependencies have known security vulnerabilities! If the count is greater than 0, this goal will exit with an non zero exit code! If you run this goal on a CI server, this will break the build if there are security vulnerabilities!

securityAndLicenseCheck

Similar to the securityCheck, this goal is checking security AND license violations!

mvn versioneye:securityAndLicenseCheck

If a dependency has a known security vulnerability OR a dependency is violating the license whitelist on the server, this goal will exit with an non zero exit code! Means, that will break your build if you run it on a CI server!

Try out this cool new features and leave a feedback comment here if you like 😉

Organisations & Teams

A big new feature was released today! The organisations & teams feature. It is replacing the “collaborations” feature. The “collaborations” feature didn’t worked out very well. It was too difficult and too time consuming to ad collaborators to the project. You had to fill the collaborators list for each project separately. That was just a lot of repeating work and many users asked for a “Team” feature, where you can define a Team once and assign a Team to a project. The new release today goes even one step further.

Organisations

Now VersionEye has a concept of organisations, just like GitHub. Now projects can belong to an organisation, instead of to a single user. An organisation can have multiple teams, license whitelists, component whitelists and projects. A user can create/belong to multiple organisations.

Screen Shot 2015-11-29 at 17.20.35

The license and component whitelists are now only available through organisations. That way it’s easier to share them with other team members.

Teams

Every organisation has by default at least 1 team. The “Owners” team. The users in the “Owners” Team are the administrators of the organisation. They can:

  • edit the organisation detail informations
  • create/delete teams
  • add users to a team
  • remove users from a team
  • create/edit/delete license whitelists
  • create/edit/delete component whitelists
  • create/edit/delete projects inside of the organisation
  • transfer project ownership to the organisation

They can not:

  • delete the “Owners” team
  • delete the last user/member in the “Owners” team. There must be always at least 1 member in that team.

Members of other teams can:

  • remove them self from the list
  • edit/delete projects which are assigned to the team.
  • see all projects inside of the organisation

They can not:

  • edit the organisations detail informations
  • create/delete teams
  • add new members to a team
  • remove other members from a team
  • create/edit/delete license whitelists
  • create/edit/delete component whitelists
  • transfer project ownership to the organisation

Transfer a project to an organisation

If you want to work on a project with multiple team members it makes sense to transfer the project to an organisation and assign a team to the project. First of all you have to create an organisation. If you do so, you are automatically added to the “Owners” team of that organisation. Now you can transfer a project to the newly created organisation. Just go the project detail page to the settings tab. At the bottom you will see a select input field with all organisations where you have admin rights.

Screen Shot 2015-11-29 at 17.35.07.png

Simply select the organisation which should be the new owner of the project and click the “Transfer” button. If you go back to the project list view where you can see all your projects, you will notice that the project disappeared. Above the project list there is a filter. The first input field is the organisation filter. By default your personal account is pre selected. But here you can select and filter for an organisation you belong to. That way you will see only projects from that organisation.

VersionEyeGroupView

Create a Team

A team is a collection of users inside an organisation which work on the same projects. Navigate to the “Team” page inside your organisation and create as many teams as you want.

VersionEyeTeams

Click on the team name to add some team members. Simply start typing the username or full name of a user and you will get some autocompletion with existing users.

VersionEyeAddTeamMember.png

Every new member of the team will receive an email notification that he was added to your team/organisation.

Now navigate to a project which you transferred already to the organisation. In the settings tab at the bottom a list of all teams of the organisation will show up. Simply select the team you would like to assign to the project and click the “transfer” button.

VersionEyeTeamAssignment.png

All the team members will receive the email notifications to the project. They all can edit the project settings and use their own API key to update the project via the API.

API

The API was updated as well. If a project is created through the API it can be assigned to an organisation at creation time.

VersionEye-API.png

The “orga_name” is optional. If it matches with an existing organisation name and the users API key has admin permissions for that organisation, the project will be assigned to that orga. Otherwise the project belongs to the user which is creating the project.

The other changes of the API are transparent for the users.

What is missing

Delivering this “Organisations/Teams” feature was a big refactoring. A lot of code had to be touched for that and the work is still not 100% done. 3 important things are still missing:

  1. Payment/Billing: Currently the organisation itself can’t have a subscription/plan. This is coming soon. The final goal is that an organisation can subscribe to plan and the whole billing is happening through an organisation, just like at GitHub.
  2. Creating Projects: If you create a new project via the API you can assign it directly to an organisation. This doesn’t work in the web interface yet. If you create a new project through the UI it always belongs to your user account and you have to transfer the project to an organisation if you want to have it in an organisation.
  3. OAuth: If a project was created through an OAuth process, for example from GitHub, Bitbucket or Stash, VersionEye needs OAuth credentials to update the project regularly. As the organisation doesn’t have OAuth credentials itself VersionEye randomly picks a team member of the project and assumes that the team member has the right OAuth credentials to update the project. In future it will be possible to pick a team member which should be used for OAuth update operations.

Feedback

This is a big new feature and many users and companies asked for it. Please test it and give constructive feedback. I would like to know how well it works for you and what else (beside the 3 points above) is missing to make it super awesome. You can leave a comment here, mention VersionEye at Twitter and simply shoot me an email.

CSV Export

At VersionEye you can setup a license whitelist to enforce a license policy. For each project there is a PDF Export, which contains the BoM (Bill of Materials). Now the same export is available as CSV as well. The links for the export are in the project detail view in the license tab.

VersionEye-License-Export

The new CSV Export has the same format as the PDF Export. It contains the list of dependencies with the information if they violate the license whitelist or not. Here is a screenshot of an example.

VersionEye-License-CSV-Export

The PDF/CSV Export also contains the current status of the license whitelist and the current status of the assigned component whitelist. That way the exported document is a complete snapshot of the current state of the project, license whitelist and component whitelist. That makes it easy to reproduce why a component is whitelisted or not.

Try out the CSV Export and let me know if you have any questions. Either here in the comments or on Twitter.