SageMaker credential exposure starts, in AWS's account, with a failed Git clone. AWS bulletin 2026-134-AWS, published October 14, 2026, says that when a clone fails in the SageMaker Python SDK's git_utils.py, repository credentials in the subprocess error output are written to CloudWatch Logs without redaction, and an authenticated remote user with access to those logs can obtain them in plaintext. The affected releases are V2 before 2.257.3 and V3 before 3.10.0; both lines are fixed. The usual response is upgrade, check the logs, rotate. This note is about the last two steps, because the bulletin does not say which logs, which credentials or which readers.
It keeps three things apart: what AWS states, what I found reading the code, and what is my own method.
What AWS Says About SageMaker Credential Exposure
AWS ties the exposure to one condition: a failed Git clone in git_utils.py. It names CloudWatch Logs as the destination and an authenticated remote user with access to those logs as the reader. Its workarounds are to avoid credentials in Git repository URLs, to restrict access to the CloudWatch Logs holding SDK output, to review logs from failed clones, and to rotate any credentials present in the affected output.
The bulletin does not name an SDK entry point, say which SageMaker environments send the text to CloudWatch, or address retention. The GitHub advisory GHSA-g6w4-jvgp-68x2 corroborates the failed-clone condition and the fixed versions, and it words the outcome more cautiously: plaintext credentials might be written to CloudWatch Logs. It also names the clone-command function in git_utils.py as the one that does not redact. The bulletin states the CloudWatch write without that qualification. I treat the two as one vendor position with two phrasings, and I do not resolve the difference. The package-level code findings below are my own.
What the Code Shows
I compared the PyPI releases 2.257.2 (the last affected V2) and 2.257.3 (the fixed one), reading the source without running it. This is my finding about SageMaker credential exposure, not AWS's.
In 2.257.2, the git_config dictionary accepts a token, or a username and password, for HTTPS repositories, and the SDK inserts it into the clone URL as https://<token>@host/... or https://user:password@host/.... The clone then runs git clone through Python's subprocess.check_call with that URL as an argument and no exception handler on the HTTPS path. The Estimator and Model classes both call this function.
Python's CalledProcessError message includes the full argument list. In a local test with a dummy token and an unresolvable host, the message read Command '['git', 'clone', 'https://tok123@invalid.invalid/r.git', ...]' returned non-zero exit status 128. On this path the credential travels in the exception text and in any traceback built from it. The module makes no logging calls of its own, so the vector is the exception, not an SDK log statement.
The fix in 2.257.3 adds a function that replaces the credential portion of the URL with <credentials-redacted> and re-raises the failed clone with the redacted command. A code comment gives the purpose: keeping plaintext tokens and passwords out of logs and tracebacks. The SSH branch of the same function does not place a credential in the command.
Two boundaries apply. I did not run SageMaker, so I have not reproduced the CloudWatch write AWS describes. And exception text goes wherever the calling process has its standard error and tracebacks captured. AWS states CloudWatch; which of your notebooks, jobs and pipeline steps send it there is a property of your environment, and this finding does not establish it for all of them. I also did not test whether git's own error output shows the URL.
V3: What I Inspected
For V3 I read sagemaker-core 2.9.0 (before the fix) and 2.10.0 (the fixed release). The sagemaker 3.9.0 and 3.10.0 packages are meta-packages that require these as minimum versions, alongside sagemaker-train, sagemaker-serve and sagemaker-mlops at 1.9.0 and 1.10.0 respectively. A fresh install of either can resolve to later releases of those packages, so what I read is the minimum each requires, not whatever a given environment resolved.
In core 2.9.0, git_utils.py holds the same credential-embedding code as V2 2.257.2, and 2.10.0 carries the same redaction fix. In core 2.9.0 and in the train, serve and mlops packages at 1.9.0, I found no call to git_clone_repo and no import of the module; the serve package only carries a git_config attribute set to None. So in the packages I scanned, no internal call path reaches the function. That is not proof that no path exists anywhere in V3. It is a public module function, so code that calls it directly would be exposed, and whether yours does is a question for your code.
That is the limit of the reading. I did not scan other V3 packages or later point releases. This is not a statement that V3 is unaffected: AWS lists V3 before 3.10.0 as impacted, the advisory gives V3 3.0.0 through 3.9.x, and for SageMaker credential exposure that listing is the one to act on.
Which Credentials May Have Entered Diagnostic Output
This section is my method for scoping SageMaker credential exposure. AWS does not prescribe it and no vendor has confirmed it. It follows from the code finding and from the bulletin's own workarounds.
-
Find the call sites. Which code passes a token, or a username and password, in
git_config, or builds a clone URL containing one, throughEstimator,Modelor direct use of the module? Which SDK version ran each? - Find where the output went. For each call site, where do the calling process's standard error and tracebacks land: CloudWatch Logs, a pipeline orchestrator, a CI job log, an error tracker? CloudWatch is the destination AWS names, not necessarily the only one.
- Find who can read it. Who can read each destination, how long it keeps events, and whether it is copied by subscription or export. An upgrade does not itself remove events already written; retention, deletion and downstream copies each have to be checked.
- Find the failed clones. On the path I read, a successful clone raises nothing, so the failed clones in the affected window are the events that can hold a credential. A credential not found in the failed-clone output you reviewed is not evidenced as exposed through that output. That is not proof it was never captured elsewhere.
Rotation scope is then the set of credentials found in failed-clone output, readable by a population you can name. That is narrower than every Git credential you have configured and wider than the one in the incident you happened to notice. If you cannot establish where the output went, the evidence does not support scoping, and in that case I would put every credential configured for an affected call site in scope. That is my risk decision, not something the code establishes. The order of rotation is a separate question, covered in Rotating Credentials Is Not Revoking Them; this note only sets the scope.
Evidence Boundaries
| Statement | Basis | Status |
|---|---|---|
A failed clone in git_utils.py writes credentials in subprocess error output to CloudWatch Logs without redaction |
AWS bulletin 2026-134-AWS | Vendor-stated; not reproduced |
| An authenticated remote user with log access can obtain the plaintext credentials | AWS bulletin | Vendor-stated |
| Affected: V2 before 2.257.3, V3 before 3.10.0 (advisory: V3 3.0.0 through 3.9.x); fixed in 2.257.3 and 3.10.0 | AWS bulletin; GitHub advisory | Vendor-stated |
| Plaintext credentials might be written to CloudWatch Logs when a clone fails | GitHub advisory | Vendor-stated, hedged |
| 2.257.2 embeds a token or username and password in an HTTPS clone URL passed as a command argument, with no handler | PyPI source comparison | My code finding |
A failed subprocess.check_call raises an exception whose text includes the argument list |
Local test, dummy values | My test |
| 2.257.3 and core 2.10.0 redact the URL in the re-raised exception | PyPI source comparison | My code finding |
No internal caller of git_clone_repo in core 2.9.0 or train, serve and mlops 1.9.0 |
Source search | My finding; scoped to those packages |
| Which environments send the text to CloudWatch, retention, downstream copies | None | Not established |
| The four-step trace and the rotation scope derived from it | Reasoning from the above | Inference |
What This Does Not Argue
This note does not claim that any environment captured a credential, or that CloudWatch captures this text everywhere. It does not claim that V3 is unaffected, or that V3 reaches the function internally. It does not involve SSH-based clones, and it makes no claim about how likely exploitation is. It takes no position on the content of the GitHub advisory.
Architect's Verdict
A failed Git clone can leave credential-bearing command details in exception text. That is the narrow, verifiable part: the credential sits in a command argument, and the exception carries the argument list.
The real problem in SageMaker credential exposure is not the upgrade. The bulletin tells you to rotate credentials that were present in the affected output; it does not determine the rotation scope for your environment. The scope comes from tracing which credentials reached diagnostic output, where that output was collected, who could read it, and which failed clones put it there. Patching closes the code path; it does not establish whether previously emitted diagnostic output remains accessible.
AWS's own workaround is to keep the credential out of the URL. Nothing in the fix tells you who could already read it.
Originally published at rack2cloud.com.
Top comments (0)