Pull request support for GitHub

77% of all our signed up users are coming from GitHub and they use VersionEye to monitor their GitHub repositories. For a while now there is a VersionEye service hook in GitHub which calls back the VersionEye API on every git push event and updates the corresponding project in VersionEye with the latest dependencies from GitHub. But now you can use VersionEye to test your changes directly in a Pull Request and see the results of those changes in the GitHub Pull Request UI. This way you can even enforce rules that would prevent a Pull Request merge if a dependency is blacklisted or it is flagged with a security vulnerability.

Given that GitHub is moving away from Service Hooks towards native WebHooks, we changed our integration to do the same. If you are logged in to VersionEye with your GitHub account you can simply click the link “Create from GitHub” and navigate to a repository where you will see all supported project files and the corresponding branches. To monitor a project file simply flip the switch directly next to it.

Screen Shot 2016-08-16 at 14.05.48

In the example above the Gemfile in the develop branch is monitored by VersionEye and it works like this: Every time a new project file is imported from GitHub, VersionEye will automatically create a web hook on that GitHub repository, which will call back the VersionEye API on each “git push” event and if the monitored files are affected, VersionEye will re-parse the files and update the view in VersionEye immediately.

The cool thing is that this also works with pull requests! Assume somebody is forking your project, adding new dependencies and sends back a pull request. In that case the VersionEye web hook kicks in and checks if some of the dependencies have security vulnerabilities or unknown licenses.  For those cases, VersionEye will respond to the pull request with  a “failed” status message  and you can see the result directly on the GitHub Pull-Request tab, like this:

GitHub-pullrequest-VersionEye-check

By clicking on the “Details” link you can see a more specific report at VersionEye for that pull request.

gh-veye-02.png

A report like this is generated on every new commit which is added to the existing pull request. It’s similar to a build at TravisCI. Now you can still merge the pull request if you like but at least you are informed about security vulnerabilities and license issues.

In the VersionEye UI you will find a new sub menu “Pull requests” in the organisation view. Here are all commits from all pull requests are listed which belong to your organisation.

Screen Shot 2016-08-16 at 14.22.50

The interesting part here is how VersionEye is combining dynamic and static analysis. Your project is checked for security vulnerabilities and licenses on each commit and each pull request but also regularly independent from that. Even if there are no commits and no pull requests for a couple days, VersionEye will check your project once a day by default and send you an email if a new security vulnerability was found for one of your dependencies.

With this new feature the developers are getting instant feedback to their changes. In real time the dependencies are analysed and the developers informed. That saves a lot of Time & Money. We think it’s important to run this kind of analysis as early in the development process as possible. If your software is already bundled and published to a binary repository it’s too late for this kind of analysis because your software might be distributed already to a high number of devices/clients. Run your analysis as early and as often as you can 😉

And by the way, that’s all open source 😉 You can run the software on your own infrastructure and it works well together with GitHub Enterprise 🙂

This is a pretty new feature. If you find a bug or you have some kind of feedback, simply open a ticket here.

Security Vulnerabilities and Licenses via the VersionEye API

VersionEye can monitor your software dependencies and notify you about out-dated versions. But that’s not all. VersionEye can help you with licenses and security vulnerabilities as well. You can setup a license whitelist for your projects and VersionEye will notify you as soon some of your dependencies are violating your license whitelist. For PHP projects you can even receive notifications about known security vulnerabilities.

Screen Shot 2015-09-20 at 15.16.34

Now the license and security vulnerabilities are available via the API as well. With this Endpoint it’s possible to fetch information about an existing project.

Screen Shot 2015-09-20 at 15.20.14

The response is a JSON object with an Array of dependencies. Now each element in the dependencies Array has a new variable called “licenses”.

VersionEye_API_licenses

If it is empty that means that VersionEye has no license information about this dependency. Otherwise it contains a list of licenses like this.

VersionEye_API_licenses_2

“on_whitelist” shows if the license is on the license whitelist or not. If the value is “null”, that means that there is no license whitelist assigned to the project. “on_cwl” stands for on component whitelist. The values are equivalent to the “on_whitelist” field.

Beside that each element in the “dependencies” Array contains a new field “security_vulnerabilities”. That is an Array of known security vulnerabilities to the dependency.

VersionEye-API-security-vulnerabilities_2

The versioneye-php package is a wrapper around the VersionEye API and it implemented this new features already in the newest version.

Now the VersionEye API is even more powerful. Take advantage of it 😉

API rate limits

Most of the functionalities of VersionEye are available via the API as well. The VersionEye API becomes more and more popular. There are already a lot of plugins and add ons build on top of the VersionEye API. You can find a list of this plugins on the API page at the bottom.

api-a397cc184c5622fb5130af1b7baf149d

In the last couple months the API responded to more than 17 Million API requests. Because of the growing traffic I introduced some rate limits.

  • 10 calls per hour for non authenticated calls.
  • 50 calls per hour for authenticated calls with an API key.

If you want to have a higher rate limit please write a email to support@versioneye.com with your use case.

NPM Module for VersionEye

Now there is a NPM Module for the VersionEye API. The versioneye-update NPM Module was developed by Onwerk, a Software Service Provider from Mannheim (South Germany). They develop web & mobile applications with Node.JS. Like this interactive Jackpot game, built with iPad, XBox KinectRaspberry Pi and NodeJS.

The Onwerk engineers like to stay ahead of cutting edge technology. They want to keep their dependencies up-to-date to get bug & security fixes ASAP into their applications. And of course they want to take advantage of new features as soon as possible.

Onwerk+NPM=VersionEyeUpdate

VersionEye has a very good Integration for GitHub and Bitbucket. If your source code is on one of this cloud SCMs, VersionEye can monitor your package.json directly via the GitHub/Bitbucket API and you get notifications about out-dated dependencies automatically.

But the use case for Onwerk is different. They do BIG Software Projects for LARGE customers and because of NDAs und German privacy laws they are not allowed to give out the source code to anybody else. That’s why they are using the VersionEye API to get notified about out-dated dependencies.

And because they wanted to automate the whole process they developed the versioneye-update NPM module, which gets executed on each build on their private Jenkins CI Server. The process looks like this:

Onwerk-VersionEye-Integration

The NPM module versioneye-update is running on each build on the Jenkins. The module is sending the current package.json file to the VersionEye API to update an existing VersionEye project. That way VersionEye nows which dependencies are used in the project right now. VersionEye will compare the version numbers from the package.json file with the newest versions in the VersionEye database to find out-dated dependencies. If there is at least 1 out-dated dependency or at least 1 license violation VersionEye will send out an email notification to the project owner and the project collaborators.

That way the whole process is automated. The engineers don’t have to execute wired commands in the console and they are not in risk to forget something. Beside that the source code stays in house. VersionEye never has access to the source code. The only file which has to be shared with VersionEye is the package.json and that doesn’t get stored on the server! After parsing it once the file object becomes a victim of the garbage collection.

Try the versioneye-update module by yourself and give feedback. Beside this NPM module there are many other Plugins and AddOns build on top of the VersionEye API. Check them out on the API site.

Merge Projects via the VersionEye API

Round about one month ago we released a new feature which supports multiple files in one project. That is useful for almost every modern software project. Some package managers, such as Bundler, Composer, CocoaPods and others, generate anyway 2 files per project. Beside that many modern software projects are splittet up in frontend and backend. And many times there are different technologies and different package managers involved to handle the open source dependencies. That’s why it makes a lot of sense for VersionEye to support multiple files per project.

In the Web Interface you can merge already existing projects into another project. And of course you can “unmerge” them if you like. Now this 2 functions are available via the API as well.

Screen Shot 2015-02-05 at 11.42.12

Assume you have 2 projects with each 1 file. A Java project (ID: 222) with a simple pom.xml file for your backend and a JavaScript project (ID: 333) with a bower.json file. But actually it’s the same project and you would like to merge them on VersionnEye to have everything together in one single view. With this call on the API you would merge them:

/api/v2/projects/222/merge/333?api_key=YOUR_API_KEY

In that case the Java project would be the parent and the JavaScript project would be the child. If you merge them the first time the order doesn’t madder. The other way around would work the same way.

If you would like to split the 2 projects again you could do it with this call:

/api/v2/projects/222/unmerge/333?api_key=YOUR_API_KEY

Easy! Right?

There is even a third new endpoint. This one:

Screen Shot 2015-02-05 at 11.55.14

This endpoint is specially for the VersionEye Maven Plugin. On a multi module project the VersionEye Maven Plugin gets executed several times and there are multiple lifecycle runs involved. Because of the maven architecture the VersionEye Maven Plugin doesn’t know always the VersionEye ID of the parent project. That’s why this endpoint got introduced, to identify the parent by groupId and artifactId. These 2 coordinates are always available in the Maven lifecycle.

Try out the new Endpoints and leave a comment if you have questions.

VersionEye – Slack

Slack is one of the fastest growing startups in the Bay Area. Slack offers a real-time messaging, archiving and search tool that facilitates better communication within a team.

Screen Shot 2014-11-04 at 11.10.41

They just closed another investment round with $120M. Congrats! 🙂

Since last week there is VersionEye – Slack integration. It enables you to get VersionEye notifications directly in Slack.

screenshot

Many Thanks to Tom Graham for the great work. You can checkout the VersionEye-Slack project here. Feedback is welcome 🙂

VersionEye API Updates

I just updated the code for the GitHub Endpoints.

Screen Shot 2014-09-15 at 09.18.53

GET /v2/github/sync

Now “/v2/github/sync” is without “enforce” parameter and it works asynchronously. It returns the status code “running” or “done”.

{ "status": "running" }

POST /v2/github/{repo_key}

“/v2/github/{repo_key}” has a new parameter “file”. In the previous version this Endpoint was only able to import a Gemfile. Now it is possible to define the project file. The “file” parameter is optional. If it’s not specified the default value is “Gemfile”.

Screen Shot 2014-09-15 at 09.24.20

DELETE /v2/github/{repo_key}

Now “/v2/github/{repo_key}” has a “file” parameter as well. It works the same way as described for “POST /v2/github/{repo_key}”.

GET /v2/github/search

This Endpoint was removed because it was kind of redundant and out of scope. After “/v2/github/sync” you can filter down your repositories with the “GET /v2/github” Endpoint.

Screen Shot 2014-09-15 at 09.31.30

Let me know if you questions or suggestions to this.