Sonatype Safety Rating

Likelihood a project contains security vulnerabilities

How is the Sonatype Safety Rating determined?

The “Sonatype Safety Rating” is generated by our experimental analysis tool and is an aggregate rating designed to estimate the likelihood of an open source project containing security vulnerabilities.

This tool leverages a variety of metrics, including the project’s rate at which it updates vulnerable dependencies (also known as Mean Time to Update, or MTTU), as well as whether the project uses open source best practices, as measured by the OpenSSF’s Security Scorecard. The Security Scorecard assesses projects’ practices like code review, signed releases, use of dependency update tools, and other similar measures, and produces a quantitative output. Further details about OpenSSF’s Security Scorecard and the checks it runs can be found on its Github repository.

Sonatype’s analysis tool combines these metrics and uses machine learning to output a scaled result that forms the basis for the Safety Rating of a project. Projects are rated on a 1-10 scale, with 1 being the least safe and 10 being the safest. The more confident the model is that a project will not contain vulnerabilities, the higher the rating. The more confident the model is that the project will contain vulnerabilities, the lower it will rate the project. The model is based on empirical research conducted by the Sonatype Research Team, where we analyzed thousands of projects and determined a high correlation between the Safety Rating and the presence of vulnerabilities, with 88% of projects scoring below 5 having existing known vulnerabilities.

Curious to know more about the technical breakdown of our new metric? Read the full 8th Annual State of Software Supply Chain Report.

F.A.Q

How do I increase the Sonatype Safety Rating of my project?

How do I increase the Sonatype Safety Rating of my project? To improve the Safety Rating of a project, there are several important steps maintainers can take.

First, minimize the total number of dependencies used in the project. And then, more importantly, ensure you keep your dependencies up to date with automated tools that both alert you to vulnerable dependencies and make it easy to update to the safest version possible (which is not always the newest). When choosing dependencies, pick those with the highest Security Scorecard first, and then those with the highest OpenSSF Criticality and libraries.io SourceRank score.

Second, implement proven open source best practices like doing code review and avoiding dangerous coding patterns in Github Action workflows. These practices and more are laid out within the OpenSSF Security Scorecard checks and in the CII-best-practices section.

To learn more about why MTTU, Security Scorecard, Criticality Score and SourceRank are important when assessing the quality and security of open source, read our latest 8th Annual State of the Software Supply Chain report.

How can I get support if I have questions about the Sonatype Safety Rating for my project?

Feel free to contact us at ossindex@sonatype.org and we will be happy to work with you on your questions.