VersionEye 2.0 started with only 3 languages. Ruby, NodeJS and PHP. Now Python was added as well!
VersionEye supports the package manger PIP. You can simply upload your requirements.txt file to get a BOM (Bill of Materials). By default the ‘requirements.txt’ file contains only the direct dependencies. I would recommend to use Pips freeze feature to lock down all transitive dependencies. That can be achieved with this command:
python -m pip freeze > frozen-requirements.txt
That command will take all direct dependencies from the requirements.txt file, resolve all transitive dependencies and create a new file frozen-requirements.txt which contains ALL (direct & transitive) dependencies with fixed versions. That’s much better then the pure requirements.txt file because the frozen version contains all dependencies which are shipped to production. That contains ALL dependencies which are necessary to run the application.
Here is an example for a Python VersionEye scan:
Try it out and let me know how you like it! Would love to hear your feedback.
If you don’t have an account yet, signup with this promotion code “Ve2Python“. That code is valid until 4th of June and gives you a 50% discount on the monthly subscription!
More than 3 years ago I discontinued VersionEye. A SaaS tool for Open Source License Compliance & Security. VersionEye 1.0 could monitor your package manager files on GitHub, notify you about outdated dependencies and show you which of your dependencies are affected by security vulnerabilities and copyleft licenses (AGPL, MPL-2.0 and so on).
I was operating it for almost 7 years and at the end I was pretty much burned out! The reason why I discontinued it are described in this blog post. Working on it was not a joy anymore. My mailbox was flooded with support emails and after 6 years pretty much everything else looked more interesting to me than OS license compliance and security. That’s why I joined LivePerson at the End of 2017. I got a new challenge, met a lot of smart people and helped to grew the LivePerson Mannheim Office from 15 to 60 people in less than 3 years.
In 2018 Alex Spinelli, former head of Alexa OS at Amazon, joined LivePerson as the new CTO. He made a lot of organisational changes and did teach us a lot about Amazon management principles. Things like writing PR/FAQs, single threaded ownership, 2 Pizza teams, mission statements, leadership principles and many more things. That was very valuable and I enjoyed learning a lot of new things.
Of course, License compliance was a topic at LivePerson as well. Software teams had to ensure that they are compliant with a company license whitelist. The tools which we had to use for that at LivePerson have been far away from user friendly and good. With a crying eye, I looked back at VersionEye. I had no interest in resurrecting VersionEye 1.0 because I still remembered all the pain points of that solution. But somehow the topic was keeping circling in my head. So I started to think about how VersionEye 2.0 should look like. Not just on a technical level, but also on a lifestyle level. How would a version 2.0 look like which I would enjoy working on and wouldn’t burn me out again? So I thought about all the points which stressed me out at version 1.0:
Customer Support: With VersionEye 1.0 I had a lot of non paying users. Less than 1% of all signed up users (52K) paid a monthly subscription. More than 99% of all signed up users did not pay a single dime but most of them caused server costs and some of them caused support.
Too much data: In 2017 VersionEye supported something like 20 package managers and crawled all repository servers for them to collect all kind of information about software libraries. VersionEye had meta information for more than 2 Million software libraries/components. Every component can have multiple versions. A specific version of a component is an “artefact”. VersionEye had meta information for more than 50 Million artefacts. That’s of course a nice vanity KPI. Kind of cool if you can tell investors that your database contains meta information for more than 2 Million software components and 50 Million artefacts! Right? But the truth is that 99% of these software components are never referenced in customer projects! 99% of them are garbage! Even my biggest customer at that time, a big software giant from Germany, did not use more than 6K unique open source components for their active Java/NodeJS software development. Keeping the database with 2 Million components up-to-date requires a lot of crawling activities and that causes again a lot of costs.
Not focused data: VersionEye 1.0 collected even information which was not relevant for open source compliance & security. For example, the crawlers collected information about the authors of the software components and about the relationships between the components. That’s all kind of nice to know but not really necessary for the core business. Customers of VersionEye have only 3 questions:
Are my dependencies compliant with a license whitelist?
Are my dependencies vulnerable?
Are my dependencies outdated?
Open sourcing: VersionEye 1.0 was fully open source. I open sourced it because I assumed it would offer more transparency and therefor bring more new customers. And I assumed I would get a lot of contributions from open source developers. Both assumptions have been wrong! I did not really get more paying customers after I open sourced everything. My existing paying customers did not really care because they did pay for the service, implementation of new features and SLAs. Open sourcing the whole thing actually caused even more support. Because all suddenly more people opened GitHub tickets. Just very few people actually implemented an improvement and send a pull request. Most people just opened a ticket and waited for an reaction. All together I had to spend more time on GitHub tickets. I needed to read them, think about it, respond to them, doing more code reviews and so on. All activities which are very time consuming.
GitHub Integration: One feature of VersionEye 1.0 was that it could monitor your package manager files directly in your GitHub repositories. For that, users had to grant VersionEye access to their GitHub accounts/organisations. VersionEye then received a token per user/orga, which could be used to fetch files from the corresponding GitHub repositories. That of course, caused a lot of background processing. But the real pain point was that these tokens expired at some point in time and then VersionEye had to reach out to the users to ask for permission again. And that caused even more background processing.
Competition: I did focus too much on competition. At that time GitHub was announcing to offer security alerts on their platform and I thought that’s the End of the game. But it wasn’t! GitHubs security database is good but not complete. And not everybody wants to host their code on GitHub to get security notifications. Of course there are more competitors in the field. But they are all kind of expensive and difficult to use. There is still much space for improvements!
So I thought about these things and how I would do it differently for VersionEye 2.0:
Customer Support: VersionEye 2.0 would be for paying customers only. People could still sign up for free and test the solution. They would maybe get 5 scans for free and then they would have to convert to paying customers. I would be happy to do support for paying customers!
Datensparsamkeit: VersionEye 2.0 would only crawl data which is relevant for customer projects. There is no reason to fill the database with meta information about 2 Million OS components if customers only use a fraction of them. That’s why VersionEye 2.0 would do crawling on demand!
Focused data: VersionEye 2.0 would collect as few data as possible and as much as necessary. Version information, license information and information about security vulnerabilities are necessary for the core business. Information about authors, transitive dependencies, meta tags/labels, description in Markdown files and so on are not necessary.
Closed source: Open sourcing the whole solution is not necessary! For some parts of the solution it makes sense to be open source. For example for API clients and so on. But the parsers, crawlers, license normalisation engine, API and WebApplication can be closed source.
GitHub Integration: A GitHub integration like in version 1.0 would not exist anymore! There might be an integration for GitHub Actions, which is a push mechanism to the VersionEye API.
Competition: I would focus more on the product and less on the competition.
VersionEye 1.0 started as a fun project but over the years it mutated to a huge software monster which sucked me in and have eaten up all my time. VersionEye 2.0 should be fun again and that means it must be lean again. In April 2020 I started to work on VersionEye 2.0! I started completely from 0. Did not reuse the old code base. I thought about the data structure and started with a new database schema. Much more focused and lightweight than the first version.
The main idea of VersionEye 2.0 is that there is one single API Endpoint where you can send your package manager files to and you get back the most important KPIs as JSON, in seconds! You can send your files via Curl, like in this example:
Curl is already installed on most Linux/Unix based machines. That means you don’t need to download an additional client. With Curl you can already integrate a VersionEye scan into your CI/CD pipeline! You just need an API Key from VersionEye.
Now VersionEye 2.0 is online since a couple months and it has the first paying customers again. It supports less package managers than the old version and has a much smaller database. It’s much more focused and much faster than the old version. And I enjoy working on it again.
If a paying customer is requesting support for a new package manager, I will take some time and implement it. I’m also fixing bugs very quickly and I’m constantly improving the license normalisation engine. But it’s all not that time & brain consuming like at version 1.0. Much less emails now and the few emails I’m getting are mostly from paying customers or prospects.
Check it out and give it a try. If you sign up, use the Promotion code “VersionEye20“. With that you get a 50% discount on the monthly subscription. Would love to hear your feedback!
VersionEye can watch your projects on GitHub, Bitbucket or Stash and notify you about out-dated dependencies. Most time you don’t work alone on a project. Most time you have other team members who work with you on a project.
With the collaboration feature you can add collaborators to your project at VersionEye and they will receive the notification emails as well, just like you do. Just navigate to your project and click the “Collaborators” tab. Here you can start typing the name or username of the user you would like to add.
VersionEye will show you some suggestions. By clicking on “Add collaborator” the selected user is added immediately as a collaborator to your project and he/she will receive email notifications to the project. The collaborators receive the email notifications to their primary email address in the same period as the you, the project owner, do.
If you would like to add somebody who is not a user at VersionEye yet, you can simply type in an email address and VersionEye will send out an invitation email immediately.
Under the menu “Shared with me”. You can see all the projects you are added as a collaborator. If you don’t like to be a collaborator in a project, simply hit the “remove” button to leave the project and you will not receive anymore notification emails to that project.
In VersionEye Enterprise there is no “remove” button. That’s a feature 😉 If you don’t want to receive the emails you have to talk to the project owner that he removes you from the list.
Up to now VersionEye sent out one notification email for each of your projects. That was not the best solution, specially for power users with 100+ projects because they got A LOT of emails. The new version only sends out 1 single email with a summary.
Assume you have 100 projects in VersionEye. 50 of them are monitored daily, and the other 50 are monitored weekly. In that case you receive now every day just 1 single email with a summary about your daily monitored projects. That looks similar like this one here.
The Dependencies column shows how many dependencies in the project are out-dated and how many are unknown. The Licenses column shows how many dependencies violet the license whitelist and for how many dependencies VersionEye doesn’t know the license.
By clicking on the project name you come to the project detail page on VersionEye, where you can see all dependencies of the project with more details.
Up to now the project overview at VersionEye was a bit cluttered, it was hard to see which project has the most out-dated dependencies. The new project overview delivers a much more clear design in a table layout.
The table is sorted by out-dated dependencies. The project with the highest number of out-dated dependencies is always on the top. Now you can see immediately which project needs most of your love 🙂
Let me know how you like and what do you think about this new view.
As developers we try to not write all code by ourselves but to rather reuse existing code in the form of libraries (or “libs” for short). A key activity for code reuse is to search and find libraries that do what you need. That’s why searching through more than 200,000 libraries is one of the key features of VersionEye.
As you might have noticed, we already improved the search results on VersionEye dramatically. Currently, we are tracking almost 300K open source libraries. Many of them have similar or even the same names. Let’s assume you do a search for “hibernate”. There are more than 400 open source packages which include the string “hibernate” in their name. But which one is the one you are looking for?
Up to now, we only ranked search results by the best match for “name”. But this was often disappointing, as the results weren’t ranked the way you would think they should be ordered.
That’s why we came up with two important changes for the search. We included the number of followers and the number of references into the search algorithm. Now, when you search for “hibernate” you will see this:
As you can see there is a perfect match ranked 3rd place, a package with the name “hibernate”. The package with the name “hibernate-core” is now ranked 1st. It ranks higher because it has more followers and more references from other packages. That’s why we assume it is more important than the perfect match. And indeed this is the package you are looking for most of the time you search for “hibernate”, right?
All the spring submodules come first, because they have a high number of followers and references. The perfect name match only ranks on place 12.
Those two small changes to the search algorithm have improved the search already dramatically. And the good thing is, you can influence the search results by your own. Every time you follow or unfollow a package on VersionEye you influence the search results.
If you have a question or an idea about any other factor that could improve the search ranking, please let us know in the comments.
We did it again 🙂 This time we have moved from Berlin to Potsdam. Potsdam is pretty close to Berlin and with the train less than 1 hour away from our old office. We were lucky enough to get free office space from our investors Hasso Plattner Ventures. They had a spare yoga room and it seems like HPV is just not that into yoga 😉 So, we are using the room as our office now.
Here it is:
What do you think? It’s kind of cool to have our own office now. Although, it was inspiring to work in a coworking space, it is also very nice to close the door anytime we want and focus on our own work. There are no distractions anymore. Plus, we are very close to our investors, the Hasso Plattner Institut and the University of Potsdam. We also have a big lake, surrounded by a forest here. Quite idyllic.
Well, we can say that we like our new office very much and we are truly grateful to our investors that they let us stay in such an amazing office for free.
On VersionEye you can follow a software library and as soon the next version comes out you will receive an email notification. We don’t spam you. We send you 1 email per day with all the notifications to the software libraries you follow directly.
Now we changed the format a little bit. In the new email format we not only show you the new version, but also in which of your projects the software libraries is used. Here is an example how the new email looks like.
Of course this only works if VersionEye is monitoring some of your GitHub repositories.
Today we want to present lat/lon. They are using our Maven Plugin to manage their Java dependencies.
lat/lon is a Germany-based consulting and software engineering company focused on Spatial Data Infrastructure (SDI), geospatial IT standards and Open Source software. lat/lon is a Principal Member of the Open Geospatial Consortium and the main developer of the degree project (http://deegree.org/ ), the most substantial implementation of OGC and ISO standards in the field of Open Source software. It builds the foundation for numerous productive components of spatial data infrastructures – in Germany, Europe and all over the world.
This new Endpoints are used by our ruby command line tool to add GitHub repositories to VersionEye. The command line tool is completely open source and on GitHub. It’s still early alpha and we are seeking for feedback. Your feedback and pull requests are very welcome 😉
Open Source License Compliance & Security 