PSA: Please be cautious because this is an excellent opportunity for taking over packages and injecting malware by malicious people.
Example: https://www.npmjs.com/package/duplexer3 which has 4M monthly downloads just reappeared, published by a fresh npm user. They published another two versions since then, so it's possible they've initially republished unchanged package, but now are messing with the code.
I'm not saying it's a malicious attempt, but it might be and it very much looks like. Be cautious as you might don't notice if some packages your code is dependent on were republished with a malicious code. It might take some time for NPM to sort this out and restore original packages.
I just tested, and it definitely looks like a troll / hack.
> duplexer3@1.0.1 install /Users/foo/Code/foo/node_modules/duplexer3
> echo "To every thing there is a season, and a time to every purpose under the heaven:
A time to be born, and a time to die; a time to plant, and a time to pluck up that which is planted;
A time to kill, and a time to heal; a time to break down, and a time to build up;
A time to weep, and a time to laugh; a time to mourn, and a time to dance;
A time to cast away stones, and a time to gather stones together; a time to embrace, and a time to refrain from embracing;
A time to get, and a time to lose; a time to keep, and a time to cast away;
A time to rend, and a time to sew; a time to keep silence, and a time to speak;
A time to love, and a time to hate; a time of war, and a time of peace.
A time to make use of duplexer3, and a time to be without duplexer3."
To every thing there is a season, and a time to every purpose under the heaven:
A time to be born, and a time to die; a time to plant, and a time to pluck up that which is planted;
A time to kill, and a time to heal; a time to break down, and a time to build up;
A time to weep, and a time to laugh; a time to mourn, and a time to dance;
A time to cast away stones, and a time to gather stones together; a time to embrace, and a time to refrain from embracing;
A time to get, and a time to lose; a time to keep, and a time to cast away;
A time to rend, and a time to sew; a time to keep silence, and a time to speak;
A time to love, and a time to hate; a time of war, and a time of peace.
A time to make use of duplexer3, and a time to be without duplexer3.
Not to mention, it’s a Pete Seeger song, the byrds just covered it. I may be wrong but I think Seeger wrote it for Judy Collins to sing.
Edit: ok nope, Seeger didn’t “write” it for Collins, she’s just another one to cover it. Here they are both doing it if you’re interested: https://youtu.be/fA9e-vWjWpw
Start posting large parts of, say, the New International Version, let me know how that goes for you.
IOW, unless it’s the King James, it is likely very much subject to take down notices. Though I’m guessing a malicious troll is much more likely to know The Byrds than they are Old Testament.
And all this is happening just as after the public release of a serious exploit which allows malicious code to do all sorts of nefarious things when it is somehow installed on the target machine. Hmm.
Given that there's hints, at least, that the problems were caused by some particular developer's actions, I wonder about the security model for package-managed platforms altogether now. If I were a big cybercrime ring, the first thing I'd do would be, get a bunch of thugs together and knock on the front door of a developer of a widely-used package; "help us launch [the sort of attack we're seeing here] or we'll [be very upset with you] with this wrench." Is there a valid defense for a platform whose security relies on the unanimous cooperation of a widely-scattered developer base?
With cases like the current one, or the leftpad incident in 2016, I'm surprised package registries still allow recycling old package names after a package was deleted. Really seems like deleted packages should be frozen forever - if the original author never recreates it or transfers ownership, then people would have to explicitly choose to move to some new fork with a new id.
But your point about pressuring or bribing package authors still stands as a scary issue. Similar things have already happened: for example, Kite quietly buying code-editor plugins from their original authors and then adding code some consider spyware (see https://news.ycombinator.com/item?id=14902630). I believe there were cases where a similar thing happened with some Chrome extensions too...
> With cases like the current one, or the leftpad incident in 2016, I'm surprised package registries still allow recycling old package names after a package was deleted.
CPAN requires the old author to explicitly transfer or mark it abandoned-and-available-to-new-owner.
For all the things wrong with perl5 (and I love it dearly but have spent enough time I can probably list more things wrong with it than the people who hate it ;) it's always a trifle depressing to watch other ecosystems failing to steal the things we got right.
This happens all the time. The new generation creates something cool because what our parents created isnt cool any more, only to fail exactly on the same spot as our parents. Only, it was already solved in the parents last version.
This goes for cloth design, cars, houses, kitchen wares and so on, as well as software.
Just look at the microwave oven earlier...
Modern microwave ovens have all adopted impractical and quirky new UIs, when the old concept of knobs was simple and worked fairly well in the first place.
My oldest one was just two dials, the second one, 15 years old had loads of buttons and stuff, really stupidly spread out, you had to press watt, minutes, seconds, start and start was not in a corner, not in top/bottom row or any other logical place so you had to search it every time. I glued a rubber piece to it so I could find it again without having to bend down and search.
Since then I have made sure the microwave has two dials, one for time, one for effect.
Remember the watercooker that had just an on & off switch?
Then came one with an option button for 80 or 100 degrees (176 or 212, in freedoms). Never knew I needed that, but that just changed my live and I can not do without it. Reason: 80 degrees water is hot enough for my needs and saves time.
Our latest has 3 buttons, with different possiblities, beebs like a maniac when ready (an option which is not unset-able) and can do things I never knew anyone would need (like keeping it at x degrees for y minutes).
I guess it is like evolution: you experiment, keep what works and get rid of all things unfit.
Packages / projects being frozen. AFAIR that's how SourceForge works/worked. I remember a few years back being baffled that I couldn't delete my own project.
But it makes sense, other projects might depend on it, so it's archived.
It's just npm that's broken. I've never used a package manager for any other language that had these kinds of issues. It's exacerbated by the massive over-reliance on external packages in JS too. `left-pad` really shone a light on how dependencies in JS land really are brought in without too much thought.
Although, I've never considered this in the case of an actual attack. It would make sense to actually fingerprint the entire source tree and record this too somewhere, so when you build it you know you are getting the right thing. Teapot basically defers this to git.
> Is there a valid defense for a platform whose security relies on the unanimous cooperation of a widely-scattered developer base?
The defense is staged deployment and active users. This obviously depends on the blutness of the malicious code.
If I may assume easily noticed effects of the malicious code: A dev at our place - using java with maven - would update the library, his workstation would get owned. This could have impacts, but if we notice, we'd wipe that workstation, re-image from backup and get in contact with sonatype to kill that version. This version would never touch staging, the last step before prod.
If we don't notice on the workstation, there's a good chance we or our IDS would notice trouble either on our testing servers or our staging servers, since especially staging is similar to prod and subject to load tests similar to prod load. Once we're there, it's back to bug reports with the library and contact with sonatype to handle that version.
If we can't notice the malicious code at all until due to really really smart activation mechanisms... well then we're in NSA conspiracy land again.
> If we can't notice the malicious code at all until due to really really smart activation mechanisms... well then we're in NSA conspiracy land again.
What about really dumb activation methods? I.e., a condition that only triggers malicious behavior several months after the date the package was subverted. You don’t have to be the NSA to write that.
What’s scary here is that there are simpleminded attacks that, AFAIK, we don’t know how to defend against.
Mh, I have a rather aggressive stance on these kind of incidents, no matter if they are availability or security related. You can fish for them, you can test for them, and there are entire classes of malicious code you cannot find. For everything you do, turing complete code can circumvent it. There's a lot of interesting reading material going on in the space of malware analysis regarding sandbox detection, for example.
So stop worrying. Try to catch as much as feasible before prod. Then focus on detecting, alerting and ending the actual incident. If code causes an incident, it't probably measurable and detectable. And even then you won't be able to catch everything. As long as a server has behavior observable from the internet, it could be exfiltrating data.
Tested restores with at most 59 minutes of data loss for prod clusters within 90 minutes after order. 30ish minutes of downtime. We could even inspect binlogs for a full restore afterwards on a per-request basis for our big customers.
Sounds like good hygiene, though it seems burdensome if everyone must do it or seriously risk infection. Ideally there would be at least minimal sanity checks and a formal process before a package can be claimed by someone else.
In case anyone was considering sending him $10, no, his hypothetical code would not be running on the Google login page. Google does not pull in external dependencies willy nilly like that.
On Google scale you quite certainly want to do that. Not just for security, but for legal reasons. You really don't want to end up using for example AGPL licensed stuff in wrong places and if you just blindly pull stuff with dependencies from package manager, this could easily happen.
One of the recent True Geordie podcasts features the "YouPorn Guy" who talks about finding it near impossible to get lawyers not on a retainer from Google to fight them.
Sure a legal audit is standard and usually much simpler than a full source audit for security, which has a complexity proportional to the project size.
I sincerely hope all modern package managers, when invoked with sudo, immediately spawn a very-low-privilege process that does most of the work sandboxed to /tmp/whatnot, and the root process just copies files to the right place and runs some system calls to update databases etc.
Most package managers I know support Turing complete install hooks. How would a package manager detect what parts of those require/are safe to run with root?
No, that's the entire point. They need almost nothing at all but the ability to run code fast in a loop with memory calls. The entire point is that they bypass privilege checks.
I'm not sure if it would help much. That means you either have to have users be able to recognize and eyeball-validate hashes ("sure, this is left-pad-5ffc991e; that's what I want! Wait, shit, it's actually left-pad-5ffd991e, never mind; wrong package), or you need pre-existing databases of trusted hashes (which either puts you right back at a registry a la NPM, or leaves you reliant on a package.lock file or similar, which doesn't cover many common use cases for secure package signing).
Detailed description what you could do with a malicious npm package is currently on he front page: "Harvesting credit card numbers and passwords from websites"
Hey, I wrote that article :) - yes it was pure coincidence, I just decided with all the security stuff going on this week (Spectre/Meltdown, I hadn't heard about the npm stuff) I'd write and article about it.
I am very surprised that a package manager of this calibre and impact abstains from best practices when it comes to authentication through code-signing. Other package managers are miles ahead of NPM. For example, Nix, which uses immutability and hashing to always produce the same artifact, regardless of changes of the sources.
So I know rpms and debs are signed, as I've setup repos for both. Docker repositories require a valid SSL key (or you have to manually allow untrusted repos). But do Python packages and Ruby gems have signature verification? How does pypy/pip and gem deal with validating a package is what it claims to be?
You have a point, but we need to take into account that the technology has been around for a long time, the risks are well known and documented, and safety concerns of most of these package managers have been addressed to maintainers.
The example in the article has come to light accidentally, but we must seriously ask ourselves how many incidents are currently unidentified.
Besides, you can use Nix for 'normal' development. It is suitable for more things than just a distro package manager.
Signing won't help unless the end user specifies the signature or certificate that they expect (signing would only help ensure package upgrades are from the same author).
If you're going to have clients specify a signature anyway, then you don't need to sign packages, you just need strong one way hash function, like SHA-1024 or something. User executes "pkg-mgr install [package name] ae36f862..."
Either way, every tutorial using npm will become invalid.
"npm install packagename" could record the public key in package.json (or package-lock.json) on first save, and only accept installs (or upgrades) matching the same public key. Just like how android app code signing works, or similar to ssh known_hosts trust-on-first-use.
Granted it wouldn't save those adding a new package to a project the first time, but it would save the bacon of anyone re-running "npm install" in an existing project, for example during a deploy, or when trying to upgrade to a newer version of a given package.
independent site that maps packages to author certs that npm uses for verification at install time?
also, this is a problem that every package mgmt system faces. they alert on changes on upgrade but there's a requirement at the end user level to verify that at install time, the cert being trusted is the right one.
I'm surprised there wasn't a global lock-down on new package registrations (or at least with the known names of lost packages) while they were working to restore them.
How does RubyGems handle a package being removed and replaced by a different (and maybe malicious) actor? Not allow a package to be deleted? Block the package name from being claimed by someone else?
> Once you've yanked all versions of a gem, anyone can push onto that same gem namespace and effectively take it over. This way, we kind of automate the process of taking over old gem namespaces.
Shit. That's a good point, I downloaded the Heroku CALI during the attack and it uses duplexer3. I got a weird message that seemed "off" during postinstall.
Example: https://www.npmjs.com/package/duplexer3 which has 4M monthly downloads just reappeared, published by a fresh npm user. They published another two versions since then, so it's possible they've initially republished unchanged package, but now are messing with the code.
Previously the package belonged to someone else: https://webcache.googleusercontent.com/search?q=cache:oDbrgP...
I'm not saying it's a malicious attempt, but it might be and it very much looks like. Be cautious as you might don't notice if some packages your code is dependent on were republished with a malicious code. It might take some time for NPM to sort this out and restore original packages.