The Front Door Was Never the Problem

In March 2024, a Microsoft engineer named Andres Freund noticed that SSH logins on a test machine were taking about half a second longer than they should. Most people would have shrugged it off. He didn't. Pulling on that thread led him to a backdoor buried inside xz utils, a compression library so unglamorous that almost nobody thinks about it, yet present on a staggering number of Linux systems. The backdoor had been planted by a contributor who had spent the better part of two years earning the trust of the project's burned-out maintainer before slipping malicious code into the release builds.

We got lucky that time. A 500-millisecond delay and one stubborn engineer were the only things standing between a quiet patch and a compromise that could have reached countless servers running production workloads. That's the part worth sitting with. The whole thing nearly worked, and it didn't fail because our defenses caught it. It failed because someone was annoyed by a slow login.

This is what software supply chain attacks look like now, and they have gotten a lot more common. Instead of attacking your application directly, someone goes after the code your application depends on: a library, a build tool, a CI plugin, an npm package three layers deep that you've never heard of and never chose. You pull it in transitively, your build server runs whatever install scripts it ships with, and now their code runs with your permissions.

How We Ended Up Here

None of this is an accident of bad luck. It's the predictable result of how modern software gets built. A typical JavaScript project can drag in hundreds or even thousands of packages once you count the dependencies of your dependencies. Each one is maintained by someone you've never met, on their own schedule, often for free. You are, whether you've admitted it to yourself or not, trusting all of them.

Attackers figured out that this trust is cheap to exploit. Typosquatting is the lazy version: register a package called reqeusts or electorn and wait for someone to fat-finger an install command. Then there's account takeover, where a maintainer's credentials get phished or reused from some old breach, and the attacker pushes a poisoned version of a package that millions of people already depend on. The 2018 event-stream incident worked exactly like that, with an attacker volunteering to take over an abandoned package and later sneaking in code that targeted a specific cryptocurrency wallet.

The more patient attackers play the long game, like the xz case, or they go after the build pipeline itself. The SolarWinds breach didn't tamper with the source code at all. The attackers compromised the build process so that the malicious code got injected during compilation, which meant the shipped binaries were poisoned even though the repository looked clean. Codecov got hit through a leaked credential in a Docker image that let attackers modify a script thousands of CI pipelines downloaded and ran without a second thought.

Why It's So Hard to Stop

The uncomfortable truth is that most teams have very little idea what's actually running inside their builds. You can read your own code line by line, but nobody reads the source of every dependency on every update. There isn't enough time in the day, and the tooling has historically made it harder than it should be.

Install scripts make this worse. In a lot of ecosystems, simply installing a package can execute arbitrary code on your machine before you've run a single line of the thing you actually wanted. Continuous integration makes it worse again, because your build servers usually have access to signing keys, deployment credentials, and cloud tokens. That's the keys to the kingdom, handed to a process that just executed code from a stranger.

So what helps? Not silver bullets, but a few habits that genuinely move the needle. Pin your dependencies to exact versions and use lockfiles, so a new release can't silently slip in behind you. Generate a software bill of materials, the SBOM everyone's been talking about, so you at least have an inventory of what you're shipping when something does go wrong. Turn off install scripts where you can, and run builds in sandboxed environments that don't carry production secrets. Scan for known vulnerabilities, sure, but understand that scanners only catch the things we already know about, and the whole point of these attacks is that we don't.

The deeper fix is cultural, and it's slower. A huge amount of critical infrastructure rests on projects maintained by one or two exhausted volunteers who get plenty of complaints and almost no support. The xz maintainer was being pressured to hand off responsibility precisely because he was overwhelmed, which is exactly the opening the attacker exploited. Throwing money, contributors, and attention at the unglamorous libraries holding everything up isn't charity. It's basic risk management, and we've been treating it as optional for too long.

Supply chain security doesn't reward heroics, and it rarely makes the news unless something goes badly wrong. But the next Andres Freund might not be paying attention to a slow login, and we shouldn't be betting the industry on the hope that they are.