LiteLLM Python Package Compromised in Sophisticated TeamPCP Supply Chain Attack, Unveiling Credential Harvesters and Kubernetes Backdoors

The prominent Python package litellm, widely utilized for interfacing with various large language model (LLM) APIs, has fallen victim to a sophisticated supply chain attack orchestrated by the notorious threat actor TeamPCP. This compromise, which saw the deployment of two malicious versions (1.82.7 and 1.82.8) to the Python Package Index (PyPI), introduced a dangerous payload encompassing a multi-faceted credential harvester, a potent Kubernetes lateral movement toolkit, and a stealthy, persistent backdoor. The incident underscores a critical escalation in TeamPCP’s ongoing campaign, which has systematically targeted high-leverage points within the software supply chain, including crucial security tools and developer infrastructure across multiple ecosystems.

Understanding the Target: The Significance of LiteLLM in the AI Ecosystem

LiteLLM serves as a pivotal open-source library, simplifying the process for developers to connect their applications to a multitude of large language models, including those from OpenAI, Anthropic, Google, and others. By abstracting away the complexities of different API integrations, LiteLLM enables seamless switching between models, managing retries, fallbacks, and even handling streaming responses. Its utility makes it a foundational component in many AI-driven applications and development pipelines, ranging from small startups to larger enterprises experimenting with or deploying generative AI solutions. The widespread adoption of LiteLLM, evidenced by its significant download numbers and active developer community, rendered it an exceptionally attractive target for attackers seeking to gain broad access to development and production environments. A compromise of such a utility provides a direct conduit into systems that process sensitive data, manage critical infrastructure, and potentially hold valuable intellectual property related to AI development, making it a high-value target for both espionage and financially motivated cybercriminals.

The Prolific Threat Actor: TeamPCP’s Escalating Campaign Against the Software Supply Chain

TeamPCP has rapidly emerged as a significant force in the realm of software supply chain attacks. Prior to the litellm compromise, the group had already garnered notoriety for successfully breaching other high-profile open-source projects and security tools, notably Trivy (a popular open-source vulnerability scanner) and KICS (Keeping Infrastructure as Code Secure). These previous attacks, like the current one, demonstrated a clear strategic focus: infiltrate foundational tools that developers rely on, thereby creating a cascading effect of compromise. Their methodology often involves leveraging existing vulnerabilities or weaknesses in CI/CD (Continuous Integration/Continuous Deployment) pipelines to inject malicious code into legitimate packages. This approach exploits the inherent trust within the software development ecosystem, where developers routinely integrate third-party libraries and tools into their projects.

What distinguishes TeamPCP, beyond their technical prowess, is their brazen communication strategy. The group actively uses platforms like Telegram to boast about their exploits, mock the cybersecurity industry, and openly declare their intentions to continue targeting security tools and open-source projects. This public posturing adds a psychological dimension to their attacks, aiming to sow distrust and fear within the developer and security communities. Their self-proclaimed goal of "stealing terrabytes [sic] of trade secrets" with "new partners" suggests a financially motivated or espionage-driven agenda, backed by a clear intent for sustained, large-scale operations. This overt challenge to the cybersecurity community highlights a growing trend of threat actors not only executing sophisticated attacks but also engaging in psychological warfare.

A Detailed Chronology of the Compromise

The timeline of the litellm compromise reveals a swift and calculated operation that highlights the rapid propagation potential of supply chain attacks. On March 24, 2026, cybersecurity researchers from multiple vendors, including Endor Labs and JFrog, publicly disclosed the discovery of two malicious versions of the litellm package, 1.82.7 and 1.82.8, on PyPI. The timing of this discovery, coupled with the nature of the payload, immediately raised alarms within the security community.

Investigations quickly pointed to a potential origin stemming from litellm‘s integration of Trivy within its CI/CD workflow. Given TeamPCP’s earlier compromise of Trivy, it is highly probable that access gained through the Trivy incident facilitated the insertion of malicious code into litellm‘s build process or directly into its source code repository. This demonstrates a sophisticated understanding of how modern software is developed and deployed, allowing the attackers to pivot from one compromised tool to another, exploiting the interconnectedness of the development pipeline.

Upon detection, PyPI administrators acted promptly to remove both backdoored versions, mitigating immediate further exposure. However, by the time of removal, these versions had likely been downloaded by numerous users, embedding the sophisticated malware into potentially thousands of development and production environments globally. This rapid detection and removal highlight the ongoing race between attackers seeking to exploit the supply chain and security researchers striving to protect it, emphasizing the critical need for continuous monitoring and rapid response mechanisms.

Anatomy of the Payload: A Three-Stage Attack for Maximum Impact

The malicious payload deployed by TeamPCP in the compromised litellm versions is a masterclass in multi-stage attack sophistication, designed for maximum impact, stealth, and persistence. Endor Labs researcher Kiran Raj provided a detailed breakdown of the intricate structure, revealing a carefully orchestrated sequence of operations.

Stage 1: The Credential Harvester
The initial phase of the attack focuses on pervasive data exfiltration. Upon execution, the credential harvester meticulously sweeps the compromised system for a wide array of sensitive information. This includes, but is not limited to:

• SSH Keys: These cryptographic keys are critical for secure remote access to servers, virtual machines, and code repositories. Their compromise grants attackers unfettered access to crucial infrastructure.
• Cloud Credentials: API keys, access tokens, and configuration files for major cloud providers such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). These credentials are goldmines for attackers, potentially allowing them to provision resources, access sensitive data stored in cloud buckets, escalate privileges, and incur significant financial costs within victim cloud environments.
• Kubernetes Secrets: Encrypted sensitive data stored within Kubernetes clusters, such as database passwords, OAuth tokens, and API keys. These secrets are essential for applications running within Kubernetes clusters, and their compromise can lead to complete cluster takeover, data breaches, and service disruption.
• Cryptocurrency Wallets: Private keys or seed phrases for cryptocurrency wallets. This directly targets financial assets, indicating a clear monetary motive for the attackers, potentially leading to significant financial losses for victims.
• .env Files: Environment variable files commonly used by developers to store configuration settings, including database connection strings, API keys for external services, and other sensitive application-specific credentials in plain text.

All harvested data is then compressed into an encrypted archive, named "tpcp.tar.gz," and exfiltrated to a command-and-control (C2) domain, "models.litellm[.]cloud," via an HTTPS POST request. The use of HTTPS for exfiltration attempts to blend in with legitimate network traffic, making detection more challenging for network monitoring tools.

Stage 2: The Kubernetes Lateral Movement Toolkit
Beyond data theft, the payload is engineered for persistent control and expanded reach within containerized environments. If a Kubernetes service account token is present on the compromised system (a common occurrence in CI/CD runners, containerized applications, or Kubernetes pods), the malware leverages it to enumerate all nodes within the Kubernetes cluster. Subsequently, it deploys "privileged pods" to each of these nodes. Privileged pods have elevated capabilities, allowing them to access the host’s underlying operating system directly, bypassing standard container isolation mechanisms. Once deployed, these pods perform a chroot operation, effectively changing the root directory for the running process to the host’s file system. This grants the attacker near-complete control over the host node, enabling them to install further malware, modify system configurations, and establish deep-seated persistence across the entire cluster.

Stage 3: The Persistent Systemd Backdoor
The ultimate goal of the lateral movement toolkit is to install a persistent backdoor on every compromised node. This backdoor manifests as a systemd user service named "sysmon.service." This naming convention is notable, as it is identical to the one used in TeamPCP’s prior compromise of Trivy, indicating a consistent operational methodology and possibly shared tooling or infrastructure. The sysmon.service is configured to launch a Python script (~/.config/sysmon/sysmon.py) at regular intervals, specifically every 50 minutes. This script reaches out to "checkmarx[.]zone/raw" to fetch a URL pointing to the next-stage payload. This modular approach allows TeamPCP to dynamically update their attack capabilities, deploy new malware, or modify their objectives as needed, without requiring a new compromise of the original package. A critical detail within this stage is the presence of a "kill switch": if the fetched URL contains "youtube[.]com," the script aborts execution. This pattern has been observed in all of TeamPCP’s previous incidents, serving as a failsafe mechanism, perhaps to avoid detection during specific periods, to deactivate the malware if the C2 infrastructure is compromised, or to avoid infecting specific targets.

Evolution of Attack Vectors: From Module Import to System-Wide Persistence

TeamPCP demonstrated an aggressive evolution in its attack vectors between the two compromised litellm versions, showcasing a rapid adaptation to maximize impact and stealth:

• Version 1.82.7: The malicious code was cunningly embedded within the "litellm/proxy/proxy_server.py" file. The injection likely occurred during or after the wheel build process, ensuring that the malicious components were part of the legitimate package distribution. This code was engineered to execute at "module import time." This means that any application or process that imported litellm.proxy.proxy_server would inadvertently trigger the payload, initiating the credential harvesting and backdoor deployment without requiring explicit user interaction beyond the standard import statement. This method relies on the victim’s application actively using the specific compromised module.

• Version 1.82.8: Represented a "more aggressive vector" according to researchers. This iteration incorporated a malicious "litellm_init.pth" file at the root of the package’s wheel. Python .pth (path configuration) files, when placed in the site-packages directory (where Python packages are installed), are automatically processed by site.py during interpreter startup. This mechanism means that the malicious logic would be executed automatically on every Python process startup within the affected environment, irrespective of whether litellm itself was explicitly imported. This grants the attacker a much broader and more immediate execution context, impacting any Python application running on the compromised system. Furthermore, the .pth launcher in 1.82.8 spawns a child Python process via subprocess.Popen. This crucial detail allows the payload to run in the background, detached from the parent process, making it significantly harder to detect, monitor, and terminate, thereby enhancing stealth and persistence. The .pth file contains a single line that imports subprocess and launches a detached Python process to decode and execute the same Base64-encoded payload, ensuring stealth and persistence.

  

Broader Implications: A Collapsing Open-Source Supply Chain

The litellm compromise by TeamPCP is not an isolated incident but rather a stark indicator of a systemic vulnerability within the open-source software supply chain. As articulated by Gal Nagli, head of threat exposure at Google-owned Wiz, "The open source supply chain is collapsing in on itself. Trivy gets compromised -> LiteLLM gets compromised -> credentials from tens of thousands of environments end up in attacker hands -> and those credentials lead to the next compromise. We are stuck in a loop." This "snowball effect" illustrates the profound interconnectedness of modern software development, where a breach in one component can trigger a cascade of compromises across an entire ecosystem. The sheer volume of credentials potentially harvested from "tens of thousands of environments" represents an enormous reservoir of access for TeamPCP, fueling future attacks.

TeamPCP’s campaign is particularly alarming due to its deliberate targeting of security tools and foundational developer infrastructure. By compromising tools like Trivy (a popular vulnerability scanner) and now litellm (a critical component for AI development), they are attacking the very mechanisms designed to protect and build software. This strategy allows them to leverage trusted channels and components to distribute their malicious payloads, bypassing conventional security controls that might flag unknown or untrusted sources. The group has successfully expanded its targeting footprint across five major ecosystems: GitHub Actions, Docker Hub, npm, Open VSX, and PyPI. This multi-platform approach demonstrates their intent to achieve maximum reach and control over diverse development and deployment environments, underscoring the universal vulnerability of the software supply chain.

Socket, another security firm, emphasized that "This is a sustained operation targeting high-leverage points in the software supply chain." TeamPCP’s public statements further amplify the gravity of the situation, with them openly mocking the state of modern security research and promising to be "around for a long time stealing terrabytes [sic] of trade secrets." Their threat to partner with "other teams to perpetuate the chaos" and target "many of your favourite security tools and open-source projects in the months to come" should be taken seriously as a declaration of intent for continued, aggressive attacks. This unprecedented level of transparency from a threat actor underscores their confidence and the significant challenge they pose to the cybersecurity community.

Recommendations for Mitigating the Threat and Strengthening Supply Chain Security

In light of TeamPCP’s escalating and sophisticated supply chain attacks, organizations and individual developers must adopt a proactive and multi-layered approach to security. The following actions are crucial for containing the threat and enhancing resilience against such pervasive compromises:

1. Pin Dependencies to Exact Versions: Avoid using broad version ranges (e.g., litellm>=1.82.0) in requirements.txt, pyproject.toml, or other dependency manifests. Instead, pin to exact, verified versions (e.g., litellm==1.82.6) to prevent automatic upgrades to potentially malicious versions. Implement automated tools to regularly audit and update dependencies after thorough security checks and vetting.
2. Implement Software Bill of Materials (SBOMs): Generate and maintain comprehensive SBOMs for all applications and projects. This provides a detailed inventory of all components, their versions, and their origins, enabling quick identification of affected systems if a component is compromised and facilitating rapid response