We can definitely say 2022 is in the books but we picked out the most interesting and worthwhile bits of news about GitHub-related “fackups” of 2022. Including, serious outages that were at the tips of everyone’s tongues and Twitter feeds, high-severity vulnerabilities, data breaches, stolen credentials (and source code itself!) from well-known brands, and hacker attacks.
Our selfish goal is pretty obvious – this article is the best answer to the question “why backup GitHub data” and we believe it will close this discussion once and for all.
Check out our review of 2022 threats and failures related to GitHub.
December 2022
GitHub status info: 6 incidents
Okta’s source code stolen after a hack on GitHub repositories
Right before Christmas, Okta, a leading provider of authentication and Identity and Access Management (IAM) solutions, admitted that its private GitHub repositories were hacked. The security incident involved threat actors stealing Okta’s source code. Luckily, despite stealing the code, attackers did not gain unauthorized access to the Okta service or customer data, confirmed the company.
Oops, it happened again. September this year, Okta subsidiary AuthO, has disclosed a “security event”. Multiple code repository archives from 2020 and earlier were obtained by unknown means from its environment. But, Okta’s problems began long before – in January and continued for months. It’s been a difficult year for the Okta security team and it looks like the Christmas break wasn’t so merry after all.
Bleeping Computer | Bleeping Computer
GitHub Actions vulnerable to Rust artifact poisoning possible
Legit Security discovered and revealed a new software supply chain vulnerability class in GitHub Actions and Rust. It enabled artifact poisoning to attack the underlying software development pipelines by replacing a legitimate artifact’s content with a malicious payload. Once in place, this infected data could have been used to perform attacks using Rust and GitHub Actions. It enabled any users to execute code in a privileged pipeline. So, a hacker could extract repo secrets and credentials, modify settings, or tamper with the source code with GitHub API use in the worst scenario. This attack highlighted the risk that insecure software pipelines pose. It also showed that this problem likely affects a huge number of open-source projects as its maintainers usually run tests on contributed code before analyzing it by themselves.
November 2022
GitHub status info: 19 incidents
Dropbox breach – hackers steal 130 GitHub repositories
It all started with phishing and human error… Dropbox disclosed a security breach after threat actors stole 130 code repositories using employee credentials stolen in a phishing attack – by faking a CircleCI email and login screen. The code accessed by this threat actor contained some credentials and API keys used by Dropbox developers along with a few thousand names and email addresses belonging to employees. Dropbox claims these code repositories were not connected to their core applications, instead that this repo contained modified third-party libraries, internal prototypes, and other internal tools. We will probably never know how much this data was worth to the company and how much the incident cost them after all.
We highly recommend a thorough analysis of our GitGuardian colleagues’ article on the Dropbox breach.
GitHub Copilot hit with a code copyright lawsuit
GitHub Copilot – a programming auto-suggestion tool trained from public source code on the internet – has been caught generating what appears to be copyrighted code. On the 3rd of November, GitHub, Microsoft, and OpenAI have been hit with a class-action lawsuit concerning alleged open-source license copyright violations arising from the use of GitHub Copilot. The plaintiffs, led by programmer and lawyer Matthew Butterick, allege that Copilot reproduces their copyrighted code without attribution or notifying users of license requirements. The 56-page suit claims Copilot “violates the licenses that open-source programmers chose and monetizes their code despite GitHub’s pledge never to do so.”
The situation is still developing… To be continued…
Data Center Knowledge | TechTarget
October
GitHub status info: 17 incidents
Checkmarx “RepoJacking” report: how to attack the software supply chain with a simple rename
Checkmarx Supply Chain Security team found a “high-severity” vulnerability in GitHub that could allow an attacker to take control over a GitHub repo, and potentially infect all apps and other code with a malicious payload. Vulnerable to this flaw were all renamed usernames on GitHub, including over 10K packages on the Go, Swift, and Packagist package managers. As a result, thousands of packages could have been hijacked and served malicious code to millions of GitHub users. RepoJacking is a technique to hijack renamed repository URLs traffic and directing it to the attacker’s repo by exploiting a logical flaw that breaks the original redirect. A GitHub repository becomes vulnerable when its owner decided to rename his username while the old username is available for registration. Luckily, the vulnerability was fixed by GitHub after unveiling it by the Checkmarx team.
Toyota data breach involving source code hosted on GitHub
On October 7th, Toyota revealed they had accidentally exposed a credential allowing access to customer data in a public GitHub repo for… almost 5 years (!). Scale? Exposed data included identification numbers and emails of over 290,000 customers. Luckily, credit card data and phone numbers were not stored in this database. As of now, there is no sign that this breach would allow bad actors to do more than just harvest emails and the associated customer management numbers. Toyota has not been able to confirm any abuse or attacks have occurred using harvested data.
September
GitHub status info: 15 incidents
The Department of Veterans Affairs investigates breach after federal contractor publishes source code
Human error – again. The Department of Veterans Affairs was conducting a cyber breach investigation after one of the contractors allegedly copied source code from a VA-managed GitHub account and published it on their own personal GitHub account and then switched to public mode. The exposed information included hard-coded admin account privileges, encrypted key tokens, and specific database table information. Moreover, this data is said to have been cloned by at least six unrelated IP addresses. On the other hand, according to VA spokesperson, the data was not administrative credentials and did not present a risk to VA or Veteran data.
Merge requests and GitHub workflows may lead to supply-chain attacks
Security researchers at Legit Security identified vulnerabilities in the GitHub automated workflows used by Google Firebase and Apache Camel that could have been abused to compromise those open-source projects through their GitHub CI/CD pipeline and insert malicious code. This exploitation technique named “GitHub Environment Injection” enables the exploitation platform’s automated integration and injecting a malicious payload into a GitHub environment variable called GITHUB_ENV. This way a compromised or rogue developer could have used it to alter the source code for Apache Camel or Firebase and conduct a supply-chain attack on this code users. As a result, this malicious code could have ended up being widely deployed.
CircleCI & GitHub phishing scam – be warned
On September 16, GitHub Security warned their users against a phishing campaign impersonating CircleCI to steal user credentials and two-factor codes. Threat actors made a fake GitHub login phishing site to steal any credentials entered – including 2FA codes (except those with hardware security keys). Then it was possible to instantly create GitHub Personal Access Tokens (PATs), authorize OAuth applications, and add SSH keys to the account to preserve access just in case the user changes their password. Threat actors could also download private repository content, including the one owned by organizations and collaborators (and then wipe it out, leak it, sell it, or blackmail the organization). They could also create new GitHub user accounts (if a compromised one had management permissions) and add them to an organization for future reference.
The list of recommendations to protect the account from GitHub included a password and 2FA codes reset and a review of PATs. GitHub recommended user hardware security keys, WebAuthn 2FA, or a browser-integrated password manager with an auto-fill option. A good security practice is also to have a GitHub backup to ensure data recovery and business continuity in the event of code wipe-out, deletion, and ransomware practices.
GitHub
August
GitHub status info: 11 incidents
July
GitHub status info: 3 incidents
GitHub Actions and Azure VMs on crypto miners target
GitHub Actions and Azure virtual machines (VMs) are being leveraged for cloud-based cryptocurrency mining. According to Trend Micro researcher, “Attackers can abuse the runners or servers provided by GitHub to run an organization’s pipelines and automation by maliciously downloading and installing their own cryptocurrency miners to gain profit easily”. It identified no fewer than 1K repositories and over 550 code samples that were mining cryptocurrency using the runners provided by GitHub. Cryptojacking groups are well known to infiltrate cloud deployments through the exploitation of a different security flaw within target systems (i.e. weak credentials, unpatched bugs, or a misconfigured cloud implementation) and it seems like GitHub was not an exception.
June
GitHub status info: 13 incidents
GitHub users beware! Thousands of tokens exposed in Travis CI logs
Travis CI, a platform for software development and testing, has exposed user data containing authentication tokens to developers’ accounts on GitHub, AWS, and Docker Hub – for a second time this year. Ok, it wasn’t GitHub’s fault but still, its users could be concerned and endangered (once they used Travis CI). The platform’s API offers access to over 770 million logs with various types of credentials of free-tier users. According to researchers at Aqua Security, it exposed tens of thousands of user tokens through an API call that allowed fetching logs in clear text when using the right log number. After analyzing around 8 million logs, the researchers found around 73K sensitive strings, that included tokens, secrets, and credentials to GitHub, AWS, and Docker Hub accounts.
May
GitHub status info: 8 incidents
GitHub, Heroku, OAuth tokens case – continued
In April 2022, GitHub has alarmed about an attack using a stolen AWS API key that has been obtained when the attackers downloaded private npm repositories by abusing a compromised OAuth token from Heroku or Travis-CI – two third-party OAuth integrators. Heroku confirmed that the attacker downloaded a subset of the Heroku private GitHub repositories from GitHub that contained some Heroku source code.
After analyzing the attackers’ behavior, GitHub concluded in early May that they were listing organizations solely with the purpose of identifying accounts and private repositories to download and clone and the attack was very targeted. And in fact, much worse than we all thought. The attackers gained access to the private GitHub repositories of dozens of organizations.
Salesforce, which owns Heroku said the company’s investigation found that the same exposed token that was used in April’s attack was reused to gain access to a database and exfiltrate the hashed and salted passwords of customer user accounts (the number remains unknown).
Salesforce also ensured that they have rotated and refreshed internal Heroku credentials and put additional detections in place. GitHub also announced that it will require developers to use obligatory two-factor authentication by the end of 2023.
April
GitHub status info: 8 incidents
March
GitHub status info: 20 incidents
An unfortunate series of GitHub outages
After a series of repeated outages, GitHub was beefing up the hardware and making improvements to its MySQL database cluster to prevent outages from reoccurring. GitHub confirmed that its service downtime over the past weeks happened due to issues affecting the “health of their database”, resulting in a degraded experience for developers and impacting the performance of a large number of their services and features during periods of peak load.
During this period people reported on social media problems with pushes and pull requests, as well as multi-hour incidents on March 16, 17, 22, and 23 that lasted between two and five hours each.
GitHub is constantly taking steps to prevent its database cluster from becoming overwhelmed with traffic across its services. Well, keeping infrastructure up and running is one of their duties resulting from the Shared Responsibility Model in which GitHub operates.
Vulnerabilities found in GitHub open-source projects could impact millions
Critical vulnerabilities were found in several popular open-source projects. Many of them can cause a supply chain attack through the continuous integration (CI) process. Vulnerabilities were found in misconfigured GitHub Actions workflows with missing proper input sanitizing. It can impact millions of potential victims and allow malicious actors to inject code into the builds through issues and comments as well as to access privileged tokens.
Of the dozens of vulnerable repositories, the most popular were: Liquibase, Dynamo BIM, Wire, FaunaDB, Kogito, Astro, and Ombi.
Open source has become a critical component in all modern apps and we can foresee that these projects will be a constant target of hacker attacks. To close up the security gaps, organizations will need to include all the principles of defense-in-depth and zero-trust approach to the entire CI/CD pipeline. They shouldn’t forget to actively prevent and mitigate risks, and to include secret scanning and GitHub backup as the final line of security.
February
GitHub status info: 6 incidents
GitHub outage impacts Actions, Codespaces, PR and Issues
February started with a worldwide outage preventing access to the website, issuing commits, cloning projects, or performing pull requests.
Rogue GitHub apps could have hijacked countless repositories
A GitHub bug could have been exploited by connected third-party apps to hijack repositories. For almost a week between late February and early March, some rogue apps could have generated installation tokens with escalated permissions that could lead to unauthorized write or administrative access to developers’ repos. In a worst-case scenario, every newly generated token during that period of time could have been elevated to grant the administrator access. However, according to Microsoft, this security blunder has been addressed before any miscreants abused the flaw.
January
GitHub status info: 7 incidents
What to do in 2023 to keep your data safe?
Well, the first recommendation that you probably won’t be able to avoid as GitHub will force you to do so by the end of 2023 is to implement 2FA (and the hardware key is something to consider).
You shouldn’t forget to actively prevent and mitigate risks. You should definitely include all the principles of defense in depth, the principle of least privilege, and the zero-trust approach to the entire CI/CD pipeline.
Don’t forget about secret scanning. GitGuardian, the secret detection platform only this year alerted about credential leaks from Samsung, Uber, Dropbox, and Nvidia. So if it happens to the biggest ones, it can happen to anyone.
Finally, if all of the security means fail, remember to have GitHub backup as the final line of DevOps security. GitHub down? Simply use Disaster Recovery technologies to instantly restore all repositories and metadata to many locations and work uninterruptedly. Also note that having DevOps backup in place is one of the fundamental requirements in all major certifications, such as SOC 2 or ISO 27001, as well as your obligation under the Shared Responsibility Model.
Useful resources:
E-books
The DevOps Guide to Backup in CI/CD
GitHub Backup Guide
Cheat Sheet
Blog posts
GitHub Backup Best Practices
GitHub Disaster Recovery and GitHub Restore – Scenarios & Use Cases
GitHub Shared Responsibility Model and Source Code Protection
Videos
GitProtect Academy
DevOps backups vs. ransomware (joint webinar with GitGuardian)
Top comments (0)