Quick take
Your publish job is green. That proves you sent the file. It does not prove anyone can install it.
Every publish step I have ever written ended at the upload. The API accepted the request, the exit code was 0, the workflow went green, and I went to lunch.
A marketplace can accept an upload and then reject it in review, hold it in a queue, or list it under a version nobody sees. All of that happens after your job has already reported success.
Ours did exactly that for three weeks. We found out because a user asked why the version was so old.
The fix is one request at the end of the same job — ask the public API what the world can actually see, and fail if it disagrees with you:
PUBLISHED=$(curl -sf "$REGISTRY_API/my-package" | jq -r .version)
[ "$PUBLISHED" = "$VERSION" ] || {
echo "uploaded $VERSION, world still sees $PUBLISHED"
exit 1
}
Note the direction. You are not asking your own pipeline whether it succeeded — it already told you, and it was wrong. You are asking a stranger.
The long version — three weeks, two marketplaces, and the wording that made the job lie — is here: Your GitHub Actions run is green. Nobody can install your VS Code extension.
I build cachly — persistent memory for AI coding assistants, over MCP. Your assistant re-reads your codebase every morning. It does not have to.
Free tier, hosted in the EU: cachly.dev
Top comments (2)
A green pipeline that shipped zero real outcomes is the exact kind of automation bug that needs an external check. CI should verify the artifact or downstream state it claims to create, not only that the workflow graph reached the last node.
"Not only that the workflow graph reached the last node" — that's the sentence I should have written.
There's a second layer I only found afterwards, and it's worth the warning: my artifact check would have passed while the artifact was still wrong. The marketplace did list it. But the extension reported its own version by looking itself up under the wrong publisher id, silently fell back to '0', and every install on earth announced itself as version 0.
So the external check has to include what the artifact says about itself, not just that it exists at the right version. curl the registry, then start the thing and ask it who it is. Two questions, not one — the second is the one I'd skipped for months.