Cyberfame — security scans & rating rules for GitHub repositories

Evaluating the security of a company's digital assets is becoming increasingly critical as businesses strive to proactively manage risk and safeguard against cyberattacks. Cyberfame cybersecurity scans and rating services are utilized to weigh an organization's networks, applications, and devices using various metrics & data points. These ratings can be employed not only for monitoring vendors' & partners' safety but additionally for assessing potential merger targets or evaluating your business' overall protection posture.

In this post, we will give a summary of the security attributes we are analysing to generate Cyberfame Security Ratings.

Introduction & Notes:

The Cyberfame algorithm is providing a scalable and fully automated way to assess the most prevalent and most frequently exploited vulnerabilities of repositories which are machine auditable.

A good (>8) Cyberfame rating does not mean a particular project is secure. There is a plethora of attacks and threat vectors we currently do not analyse or haven't detected yet.
We analyse Repositories for the following attributes to compute a security rating.

Vulnerability checks
This check determines whether the project has open, unfixed vulnerabilities using the OSV (Open Source Vulnerabilities) service.

An incident case: An open vulnerability is readily exploited by attackers and should be fixed as soon as possible.

Risk: High (known vulnerabilities)

This check determines whether the project requires code review before pull requests (merge requests) are merged. Reviews detect various unintentional problems, including vulnerabilities that can be fixed immediately before they are merged, which improves the quality of the code. Reviews may also detect or deter an attacker trying to insert malicious code (either as a malicious contributor or as an attacker who has subverted a contributor’s account), because a reviewer might detect the subversion.

Remediation steps

• If the project only has one contributor or does not have enough reviewers to practically require that all contributions be reviewed, try to recruit more maintainers to the project who will be willing to review others’ work.
• Ideally, at least some of these people should be from different organizations.
• Follow security best practices by performing strict code reviews for every new pull request/merge request.
• Make “code reviews” mandatory in your repository configuration.
• Enforce the rule for administrators/code owners as well.
• Enforce the rule for administrators/code owners as well. ( Instructions for GitHub.)

Risk: High (malicious code injection)

This check determines whether the project has generated executable (binary) artifacts in the source repository, which increases user risk. Many programming language systems can generate executables from source code (e.g., C/C++ generated machine code, Java .class files, Python .pyc files, and minified JavaScript). Users will often directly use executables if they are included in the source repository, leading to many dangerous behaviours.

Problems with generated executable (binary) artefacts include: binary artefacts cannot be reviewed; generated executables allow the executable generation process to atrophy; and over time, included executables might not correspond to the source code.

Get more cyberfame if: Files in the source repository that are simultaneously reviewable source code and executables; source code in the source repository generated by other tools; and generated documentation in repositories.

Remediation steps

• Remove the generated executable artefacts from the repository.
• Build from source.

Risk: High (non-reviewable code)

This check determines whether a project’s default and release branches are protected with GitHub’s branch protection settings. Branch protection allows maintainers to define rules that enforce certain workflows for branches, such as requiring review or passing certain status checks before acceptance into a main branch, or preventing rewriting of public history.

Note: The following settings queried by the Branch-Protection check require an admin token: DismissStaleReviews, EnforceAdmin, StrictStatusCheck and RequireCodeownerReview. If the provided token does not have admin access, the check will query the branch settings accessible to non-admins and provide results based only on these settings. Even so, we recommend using a non-admin token, which provides a thorough enough result to meet most user needs.

Different types of branch protection protect against different risks: Require code review; Prevent force push; Require status checks Although requiring code review can greatly reduce the chance that unintentional or malicious code enters the "main" branch, it is not feasible for all projects

Remediation steps

• Branch protection settings should be enabled in your source hosting provider to avoid force pushes or deletion of important branches.
• For GitHub, for remediation check here.

Risk: High (malicious code injection)

This check tries to determine if the project runs tests before pull requests are merged, currently limited to repositories hosted on GitHub. Running tests helps developers catch mistakes early on, which can reduce the number of vulnerabilities that find their way into a project.

Remediation steps

• Check-in scripts that run all the tests in your repository.
• Integrate those scripts with a CI/CD platform that runs it on every pull request (e.g. if hosted on GitHub, GitHub Actions, Prow, etc).

Risk: Low (unknown unknowns)

This check tries to determine if the project has recent contributors from multiple organizations (e.g., companies). It is currently limited to repositories hosted on GitHub, and does not support other source hosting repositories (i.e., Forges).

The check looks at the Company field on the GitHub user profile for authors of recent commits. To receive the highest score, the project must have had contributors from at least 3 different companies in the last 30 commits; each of those contributors must have had at least 5 commits in the last 30 commits.

Note: Some projects cannot meet this requirement, such as small projects with only one active participant, or projects with a narrow scope that cannot attract the interest of multiple organisations. We are working on a solution for this mainly economic problem.

This check also determines whether the project’s GitHub Action workflows have dangerous code patterns, and awards the highest score when all workflows avoid these patterns. Some examples of dangerous code patterns are untrusted code checkouts, logging GitHub context and secrets, or the use of potentially untrusted inputs in scripts.

The following specific patterns are checked: Untrusted Code Checkout (misuse of potentially dangerous triggers), Script Injection with Untrusted Context Variables (inline script may execute untrusted input from attackers).

Risk: Critical (vulnerable to repository takeover)

This check tries to determine if the project uses a dependency update tool, specifically one of: dependabot, renovatebot, Sonatype Lift, or PyUp. Out-of-date dependencies make a project vulnerable to known flaws and prone to attacks. These tools automate the process of updating dependencies by scanning for outdated or insecure requirements and opening a pull request to update them if found.

This check can determine only whether the dependency update tool is enabled; it does not ensure that the tool is run or that the tool’s pull requests are merged.

Remediation steps

• Dependabot can be enabled for forks where security updates have ever been turned on, so projects maintaining stable forks should evaluate whether this behavior is satisfactory before turning it on.
• The Cyberfame App has support to migrate dockerfiles’ dependencies from version pinning to hash pinning without additional manual effort.

Risk: High (possibly vulnerable to known flaws)

This check tries to determine if the project uses fuzzing by checking if the repository name is included in the OSS-Fuzz project list, if ClusterFuzzLite is deployed in the repository, or if there are user-defined language-specified fuzzing functions.

Fuzzing, or fuzz testing, is the practice of feeding unexpected or random data into a program to expose bugs.

Remediation steps

• Regular fuzzing is important to detect vulnerabilities that may be exploited by others, especially since attackers can also use fuzzing to find the same flaws.

Risk: Medium (possible vulnerabilities in code)

This check tries to determine if the project has published a license. It works by checking standard locations for a file named according to common conventions for licenses.

A license can give users information about how the source code may or may not be used. The lack of a license will impede any kind of security review or audit and creates a legal risk for potential users.

This check will detect files in the top-level directory with any combination of the following names and extensions: LICENSE, LICENCE, COPYING, COPYRIGHT and .html, .txt, .md.

It will also detect these files in a directory named LICENSES. (Files in a LICENSES directory are typically named as their SPDX license identifier followed by an appropriate file extension, as described in the REUSE Specification).)

Remediation steps

• A license should be created in a .txt, .html, or .md file named LICENSE or COPYING.
• If a project is archived, it receives the lowest score.
• If there is at least one commit per week during the previous 90 days, the project receives the highest score.
• If there is activity on issues from users who are collaborators, members, or owners of the project, the project receives a partial score.

Risk: High (possibly unpatched vulnerabilities)

This check tries to determine if the project is published as a package, currently limited to repositories hosted on GitHub. Packages give users of a project an easy way to download, install, update, and uninstall the software through a package manager. The check looks for GitHub packaging workflows and language-specific GitHub Actions that upload the package to a corresponding hub.

You can create a package in several ways: many program language ecosystems have a generally-used packaging format supported by a language-level package manager tool and public repository; many operating system platforms also have at least one package format, tool, and public repository; or using container images.

Remediation steps

• Publish your project as a downloadable package, e.g., if hosted on GitHub, use GitHub’s mechanisms for publishing a package.
• If hosted on GitHub, use a GitHub action to release your package to language-specific hubs.
• Risk: Medium (missing security updates)
• This check tries to determine if the project pins dependencies used during its build and release process, in order to reduce security risks.
• The check works by looking for unpinned dependencies in Dockerfiles, shell scripts, and GitHub workflows
• Pinning dependencies can inhibit software updates, so it’s important to use automated tools to notify applications when their dependencies are outdated.
• All dependencies should be declared with specific versions in the package format file. * For C/C++, the code should be checked from a trusted source, and a README on the specific version used should be added (including archive SHA hashes).
• If the package manager supports lock files, they should also be checked into the source code.
• Dependencies in Dockerfiles used for building and releasing projects should be pinned by hash.
• For GitHub workflows used in building and releasing your project, pin dependencies by hash. Risk: Medium (possible compromised dependencies)

This check tries to determine if the project uses Static Application Security Testing (SAST), also known as static code analysis. It is currently limited to repositories hosted on GitHub, and does not support other source hosting repositories (i.e., Forges).

SAST is testing run on source code before the application is run. Using SAST tools can prevent known classes of bugs from being inadvertently introduced in the codebase. The checks currently look for known Github apps such as CodeQL (github-code-scanning) or SonarCloud in the recent (~30) merged PRs, or the use of “github/codeql-action” in a GitHub workflow. It also checks for the deprecated LGTM service until its forthcoming shutdown.

Note: A project that fulfils this criterion with other tools may still receive a low score on this test. There are many ways to implement SAST, and it is challenging for an automated tool like Cyberfame to detect. A low score is therefore not a definitive indication that the project is at risk.

Risk: Medium (unknown bugs)

The check looks for a file named SECURITY.md (case-insensitive) in a few well-known directories to determine if the project has published a security policy. A security policy can give users information about what constitutes a vulnerability and how to report one securely so that information about a bug is not publicly visible.

This check examines the contents of the security policy file awarding points for those policies that express vulnerability process(es), and disclosure timelines, and have links (e.g., URL(s) and email(s)) to support the users.

Remediation steps

• A security policy file called SECURITY.md should be placed in the root directory of the repository to make it easily discoverable by a vulnerability reporter.
• The file should contain information on what constitutes a vulnerability and a way to report it securely, such as an issue tracker with private issue support or an encrypted email with a published public key.
• Following coordinated vulnerability disclosure guidelines will help maintainers respond appropriately to any disclosures.

Risk: Medium (possible insecure reporting of vulnerabilities)

This check tries to determine if the project cryptographically signs release artefacts, currently limited to repositories hosted on GitHub. Signed releases attest to the provenance of the artefact.

This check looks for specific filenames in the project’s last five release assets, and awards points based on what is found.

Remediation steps

• Publish the release.
• Generate a signing key.
• Download the release as an archive locally.
• Sign the release archive with this key (should output a signature file).
• Attach the signature file next to the release archive. * If the source is hosted on GitHub, check out the steps here.

Risk: High (potential for forged releases)

This check determines whether the project’s automated workflows tokens are set to read-only by default, and is currently limited to repositories hosted on GitHub. Setting token permissions to read-only follows the principle of least privilege, which is important because attackers may use a compromised token with write access to push malicious code into the project.

The highest score is awarded when the permissions definitions in each workflow’s YAML file are set as read-only at both the top level and run level. One point is reduced from the score if all jobs have their permissions defined but top-level permissions are not defined. Additionally, points are reduced if certain write permissions are defined for a job.

Remediation steps

• Set permissions as read-all or contents: read as described in GitHub's documentation.

Risk: High (malicious code additions)

The check determines if the webhook has a token to authenticate requests.

Remediation steps

• Check whether your service supports token authentication.
• If there is support for token authentication, set the secret in the webhook configuration. See Setting up a webhook
• If there is no support for token authentication, consider implementing it by following directions you can find here: https://docs.github.com/en.

Cyberfame security ratings are an invaluable resource to people and businesses alike, allowing them to gain insight into their digital safety levels so they can make informed decisions on how best to safeguard themselves against cybercrime. Cyberfame allows you to harness a comprehensive cybersecurity rating, which evaluates factors like deep source-code analysis, software supply chain and bill of materials ratings. With this robust system in place, your business will be shielded from the latest threats with confidence. With this assessment, you're able to gain an understanding of the security of your online presence and can act on it quickly. Moreover, by taking advantage of your cyber security score, not only are you protecting yourself but also creating trust in supply chain networks - a vital part of success today where reputation is key.

Visit https://cyberfame.io today to analyse and improve your supply chain security posture.