Pull request support for GitHub

77% of all our signed up users are coming from GitHub and they use VersionEye to monitor their GitHub repositories. For a while now there is a VersionEye service hook in GitHub which calls back the VersionEye API on every git push event and updates the corresponding project in VersionEye with the latest dependencies from GitHub. But now you can use VersionEye to test your changes directly in a Pull Request and see the results of those changes in the GitHub Pull Request UI. This way you can even enforce rules that would prevent a Pull Request merge if a dependency is blacklisted or it is flagged with a security vulnerability.

Given that GitHub is moving away from Service Hooks towards native WebHooks, we changed our integration to do the same. If you are logged in to VersionEye with your GitHub account you can simply click the link “Create from GitHub” and navigate to a repository where you will see all supported project files and the corresponding branches. To monitor a project file simply flip the switch directly next to it.

Screen Shot 2016-08-16 at 14.05.48

In the example above the Gemfile in the develop branch is monitored by VersionEye and it works like this: Every time a new project file is imported from GitHub, VersionEye will automatically create a web hook on that GitHub repository, which will call back the VersionEye API on each “git push” event and if the monitored files are affected, VersionEye will re-parse the files and update the view in VersionEye immediately.

The cool thing is that this also works with pull requests! Assume somebody is forking your project, adding new dependencies and sends back a pull request. In that case the VersionEye web hook kicks in and checks if some of the dependencies have security vulnerabilities or unknown licenses.  For those cases, VersionEye will respond to the pull request with  a “failed” status message  and you can see the result directly on the GitHub Pull-Request tab, like this:

GitHub-pullrequest-VersionEye-check

By clicking on the “Details” link you can see a more specific report at VersionEye for that pull request.

gh-veye-02.png

A report like this is generated on every new commit which is added to the existing pull request. It’s similar to a build at TravisCI. Now you can still merge the pull request if you like but at least you are informed about security vulnerabilities and license issues.

In the VersionEye UI you will find a new sub menu “Pull requests” in the organisation view. Here are all commits from all pull requests are listed which belong to your organisation.

Screen Shot 2016-08-16 at 14.22.50

The interesting part here is how VersionEye is combining dynamic and static analysis. Your project is checked for security vulnerabilities and licenses on each commit and each pull request but also regularly independent from that. Even if there are no commits and no pull requests for a couple days, VersionEye will check your project once a day by default and send you an email if a new security vulnerability was found for one of your dependencies.

With this new feature the developers are getting instant feedback to their changes. In real time the dependencies are analysed and the developers informed. That saves a lot of Time & Money. We think it’s important to run this kind of analysis as early in the development process as possible. If your software is already bundled and published to a binary repository it’s too late for this kind of analysis because your software might be distributed already to a high number of devices/clients. Run your analysis as early and as often as you can 😉

And by the way, that’s all open source 😉 You can run the software on your own infrastructure and it works well together with GitHub Enterprise 🙂

This is a pretty new feature. If you find a bug or you have some kind of feedback, simply open a ticket here.

Open Ticket System for VersionEye

We at VersionEye are always interested into your Feedback! Right from the beginning the feedback icon was part of the main menu at VersionEye.

Screen Shot 2015-02-12 at 10.34.33

By clicking on the feedback icon a modal dialog appeared where you could send a message to the VersionEye Team. We received your feedback as email and it was not visible to anybody else. Many times we received a bug/feature report multiple times. Simply because many users found the same issue.

A couple days ago we changed the behaviour of the feedback icon. Instead of opening a modal dialog it redirects the user to the ticket system on Bitbucket.

Screen Shot 2015-02-12 at 10.37.10

Everybody can sign up for free at Bitbucket and open a ticket for VersionEye. The tickets are visible for everybody! Even for users who are not singed up a Bitbucket!

Why Bitbucket? Why not GitHub?

The code for VersionEye is splittet up in over 10 git repositories. For long time we hosted everything on GitHub and we used internal the GitHub ticket system for ourselves. There are 2 reasons why we moved some repositories to Bitbucket.

  • On Bitbucket you can make the ticket system public and at the same time keep the source code private.
  • The Ticket system on Bitbucket is a little bit more advanced. Specially the “vote” feature is very useful for communities. That way users can vote for a ticket and we can sort the tickets by votes. That is very useful for new feature requests. Now we don’t have to guess anymore which feature to implement first. We simply implement the features first with the most votes!

Screen Shot 2015-02-12 at 11.38.42

We still have code on GitHub and we are still paying customers of GitHub. Simply because the performance of GitHub is better. A ‘git push’ or ‘git checkout’ works on GitHub always a couple seconds faster than on Bitbucket.  With git you can have multiple backends without a problem 🙂 That’s why we all love git!

We hope you will embrace our new open ticket system and we will see many features requests and votes in future 😉

New license crawler finds licenses in git repositories

VersionEye is crawling a whole bunch of package managers + some GitHub Repositories. Many open source libraries provide their license directly within the package manager. On search.maven.org the license information is even mandatory. If somebody submits a new package and the license information is not included in the pom.xml file the package will be declined and not published. That’s a good thing! Unfortunately at most other package managers the license is not mandatory and so it comes that we have many open source libraries in our database without a license 😦

opensource_logo

But many projects on GitHub provide a LICENSE file. Other projects copy & paste the license text directly into their README page. That’s why I developed a new crawler which is looking for a LICENSE file in the root of a git repository. If some kind of LICENSE file was found the crawler tries to recognize the license text. Is it an MIT text, a GPL 3 text or some other license text? If the crawler recognizes the license text it saves the information into the VersionEye database. Otherwise not.
If there is no LICENSE file the crawler tries to find some license text directly in the REAMDE file. If the crawler couldn’t find anything it skips the repo and goes on to the next one.

The license crawler doesn’t crawl whole GitHub. It crawls only git repositories of open source packages which are already in the VersionEye database and have a corresponding link to GitHub. It basically completes meta data to already existing open source packages in the database. And right now there are 463.873 open source libraries in the VersionEye database.

Results

In the past couple days the license crawler was able to find license information for 22.694 open source libraries. 5.684 of this open source libraries are JavaScript libraries registered on Bower, without any license information. One popular example is angular-mocks. The repository provides a bower.json file without a license! The license text of angular-mocks is directly embedded into the README.md file. VersionEye found & recognized the license as MIT and displays it now on the corresponding VersionEye page. There are many more examples like this one.

Next Step

These works already pretty good. But the next logical step is to write a crawler which recognizes licenses in source code. That requires a bit more computing power and a bit more time to write it. Nothing for 2014. But a good goal for 2015 😉

Update Single Git Repository

VersionEye has a very good integration for GitHub, Bitbucket and Stash. It can monitor project files from different repositories and multiple branches. If VersionEye is connected to a GitHub Account it fetches meta data about that account via the official API. Meta data such as repo names, branch names and file names. If a branch gets added/removed from a repository this meta data is out-of date. Until now you had to hit the “Reimport all repositories” link to reimport ALL repositories with current meta data. If you have many repositories that is a bit time consuming and annoying.

Now we added in each repository view a link “Update meta-data about this repository”, which only updates branches and file names of the current repository. That’s much more efficient 🙂

UpdateSingleGitRepo

New Single Page App for GitHub/Bitbucket integration

Yesterday we launched the new Single Page App for the VersionEye – GitHub/Bitbucket integration. For sure you know the old one. This here.

OldSPA_Github

It had a lot of disadvantages. The UI was confusing and cluttered with a lot of information, which is not important for the VersionEye use case. And specially if you had a lot of branches and project files in a repository you had to scroll in the little fixed div area. That was not very comfortable.

The new SPA (Single Page App) is a completely rewrite, with ReactJS.  The UI is much simpler, there is nothing you can do wrong. This is the initial page.

Screen Shot 2014-07-31 at 10.10.40

You get ALL your repositories listed, without any other useless information. With the text input field you can filter the list and quickly get to your desired repository. By clicking on a repository name you come to the next page, there we import ALL the branches and the supported project files.

Screen Shot 2014-07-31 at 10.12.36

Here you can take advantage of the full size of your screen. If you want to monitor a file, simply flip the switch on. After the file is parsed the file name will turn into a link which leads to the VersionEye project page, where you can see all the dependencies of the file.

This SPA is much simpler than the old one. There is nothing you can do wrong and it works much faster. Beside the SPA we also refactored the backend services, which now take more advantage of RabbitMQ. You can try the new SPA in the login area here for GitHub:

https://www.versioneye.com/user/projects/github_repositories

And here for Bitbucket:

https://www.versioneye.com/user/projects/bitbucket_repositories

Let me know how you like it. Either here in the comments or on Twitter.

Switch for all repos

We have updated our GitHub Single Page App. So far we only supported repositories from languages VersionEye already supports. That means we used to read the repository language via GitHub API and if it was one of our supported languages, we showed an active “switch” widget you could turn on or off. Unfortunately, this caused errors sometimes. So we changed the behavior. Now the “switch” widget is always active. You can turn it on in each and every case and VersionEye will try to find a project file in the root of your repository.

Image

GitHub Single Page App

VersionEye stands for Continuous Updating. Therefore we did some more work on our GitHub Single Page App. The layout looks a little different and you are able to see more repositories on your screen. With a single click you can display the branches and with one more click you can add them to your VersionEye project list.

Beside this we changed the sorting once again, descending to the last commit in the repositories.

Let us know what you think!

Image